Do Filmulator and PhotoFlow support HDR DNG?
http://filebin.net/ebwcyep3gj
I’m especially curious about how Filmulator’s tone-mapping operator would deal with it.
For reasons of “I need to come up with a good interface” there are several things I plan to, but have not yet, implement in Filmulator.
Among these will be combining multiple exposures.
- HDR merging. This isn’t going to be arbitrary masking, it’ll be ideal-minimum-noise merging; a brighter exposure gets more weight than a darker exposure, as long as the brighter exposure isn’t clipped on that pixel. The CLI version of Filmulator was able to do this but it is incredibly cumbersome to use, and it was I think too conservative about avoiding clipping. You will be able to do all one brightness (stacking), or you would be able to do bracketed exposures. And it ought to be able to handle misaligned images too.
- Panorama merging. Hugin…works well, but it could use a completely rethought interface, IMO, possibly involving adding one image at a time so that you can ensure its fit with respect to previous images is “sane”. Plus, with modern computers have massive amounts of ram, Hugin’s method of writing everything out to disk is unnecessarily slow. It could use some optimization, at least.
@Morgan_Hardwood: If LibRaw and Exiv2 support it, Filmulator supports it. Seems like LibRaw doesn’t like those. It complains about the dimensions being too big…is it embedding separate image planes into one DNG, or is it merging them into one image plane like how I described in (1) above? The latter should be easily handled by Filmulator’s tonemapping algorithm.
Do note that for now I don’t really have error handling everywhere; it currently crashes when you try to import them.
Not yet for PhotoFlow, although this and OpenEXR support is in my future plans. Particularly since I’m presently modifying several tools to work in unclamped (and optionally linear) RGB mode, which is more suited for scene-referred editing, and to do unclamped colorspace conversions…
Looks like LibRaw won’t support HDR DNG in the foreseeable future…
Hmmm…it’d probably be easier to use 16 bit integer tiffs as if they’re a logarithm.
Ah yes, I should add “lack of floating-point DNG support” to the list of reasons not to use libraw if RawTherapee turns away from dcraw.
@CarVac it’s a single plane plain old not-demosaiced image with floating point precision. Obeys the DNG standard, I’m told.
PhotoFlow has recently switched to RawSpeed for the RAW decoding, and as far as I understand RawSpeed allows opening HDR DNGs, but I should make some test to see if that is really the case.
@Morgan_Hardwood Uhmmm… looks like I was wrong: RawSpeed does not recognize your files, and also Darktable does not want to open them.
However, I can rather easily switch to RawTherapee’s RAW loading backend for files that are not properly handled by RawSpeed, like in this case. Would you consider this an acceptable solution?
In fact, I think that RAW file loading in the OpenSource world is still quite a mess 
There are several courses of action here for Filmulator.
The first one is something I’ll do regardless: I’ll catch the crashes and not accept floating point DNG files, and instead simply offer stacking and bracket blending within Filmulator to achieve the same purpose.
The next is to accept high dynamic range non raw formats like EXR and a logarithmic integer tiff and maybe floating point tiff as well, and also have them be available as outputs from the initial loading stage for stitching. I’m hesitant but at the same time likely to do this. It’ll be effective and easier than completing built in stitching, but in terms of the program structure, inelegant and kludgy.
The last is to switch raw loading libraries. This might take some time, because right now I rely on LibRaw to perform demosaicing. My Halide implementation of LMMSE isn’t finished (it gives valid output but needs optimization), and I need to figure out highlight recovery, but once I do, that will give a large boost in performance especially when we get gpu acceleration working.
@Carmelo_DrRaw as a user any method is fine as long as it works 
@CarVac I don’t understand your stance - willing to support messy log and fp TIFFs which could be anything while not wanting to support fp DNG covered by the DNG Version 1.4 specification. But then you did write, [quote=“CarVac, post:4, topic:688”]
Filmulator is first and foremost for me to use.
[/quote]
@Morgan_Hardwood
Talking about HDR and floating-point formats, I am currently working on an experimental branch that not only supports reading and writing unclamped floating-point TIFFs, but also implements unclamped (i.e. HDR) editing for several tools (for example you can apply a negative exposure compensation to a floating point TIFF image to recover the details that were originally >1.0).
There I plan to include EXR and HDR DNG support as well (at least for reading).
This seems to be particularly interesting for scene-referred editing in the VFX industry, but probably it can be significantly useful for more “normal” photographers as well…
Sure! I’ll try to prepare some nice screenshot to show how the interface looks like, and few sentences to summarise the main features. This independently of a possible dedicated talk.
Thanks!
It’s a matter of code cleanliness and performance consistency. I don’t want cascades of different raw handling libraries that won’t necessarily give equivalent results.
You’re right about hacked log tiff being a bad idea though, I’ll completely scratch that.
The whole file format situation we’re in is appalling. As with many things in life, you can make a triangle out of this situation. In this case, in the corners you have: image storage options, metadata support, and licence situation. You can have a great format like BPG which supports various storage options (lossy, lossless, various bit depths, etc), great metadata support, but you can’t use it because patents. You can have OpenEXR, no licencing issues, good image storage options, but doesn’t support Exif, IPTC nor XMP (as far as I know - please prove me wrong!). Then there’s a bunch of formats with free licences and support for metadata, but image storing options from the dark ages. Ugh. There’s a bunch of good technologies which aren’t being widely used because some company demands exorbitant licencing and royalty fees, so they’d rather sink and take our generation down with them instead of facing facts and releasing the technology openly because they won’t make a dime off of it anyway. Oof.
Nice thing about HDR DNG is that it has all the benefits of typical DNG - great metadata support, various storage options, no licencing issues I know of, the files are very small - a 16-bit floating point HDR DNG made from five 12MB raw photos ends up taking about 14MB - so much space saved! And yes, 16-bit with floating-point precision is all you need.
But why should I need to deal with HDR DNG at all, unless cameras start outputting that natively?
Filmulator will support merging within the application, so the only benefit of HDR DNG is that the second time I work on it it starts up a bit faster.
I’m not going to delete the raw files, so there’s no space saved by saving any merged version aside from the final output.
- What justifies their existence considering what’s already on the libre graphics market?
Eow, here goes my first post, in a topic for which I have a certain interest…
I’ve spent the last year after resurrecting my photography hobby (into the digital age, what a head-hurter…) trying the various raw processors and generally being less than fully satisfied, 'cept for two: 1) dcraw, just does one thing and does it well, and 2) my in-camera firmware that produces JPEGs that look okay enough to which I don’t feel compelled to do too many subsequent manipulations. For feedback to the participants, I tried photoflow but my primary computer is a Windows tablet and I struggled with getting the earlier releases to run; until now, have not heard of Filmulator but I’ll give it a look.
Anyway, I’ve developed my own sense of what to do with my photos, and none of the tools did it all simply, so I started writing my own. Yes, yet another one, but I’m not that concerned about adding it to the ‘market’, so to speak. I’m probably going to put it on github in the next couple of weeks, and the only mode of support tendered will be gitub/fork. It uses FreeImage as its base, and responds to the following needs I couldn’t meet with any other software:
-
I hate sidecar files. So, my software saves the modification commands in the UserComment exif tag, like this:
rawproc-0.1 DSG_3890.NEF gamma:2.2 curve:16.0,0.0,64.0,109.0,131.0,194.0,220.0,256.0 saturation:17
It also has a “Open source…” menu selection that will find this tag, open the source file and preload the processing commands. -
I wanted one tool to quickly get from .NEF to internet-ready .JPG. Funny, thing, I found the processing that hardest to do in this regard is resize. After doing just a few things to my 16-bit image, I simply want to make 640xXXX images for posting, and that last operation is either 1) non-existent, or 2) full of parameters from which to choose.
-
I wanted a tool that would work on my cheap 32-bit Windows tablet. I bought a Microcenter Winbook tablet to read when I travel, and for photography to do what I call ‘contact sheet’ processing in the field. Most of the GUI applications are really to ‘heavy’ to run on a 2G 32-bit machine, and the UIs aren’t touch-friendly. Ironically, I found the Windows 8/10 Photos application is really pretty handy in this use, but I’ve only edited my camera JPEGs with it and I’d rather move to something I know is working in the 16-bit range. Writing touch-friendly software is a bit of a challenge right now, having to make my own sliders and toolbars, but it keeps me out of the bars…
-
I don’t want image management. In that regard, Photoflow was the closest thing to scratching that itch. I’ll use the OS file explorer to manage my images, thank you. And, referring back to #3, I found that software that included image management was a challenge to run on my cheap hardware. I envision my field workflow to go something like this: 1) copy my NEFs to a directory. 2) scan the thumbnails for the good ones, and make TIFs from them with basic operations to render a usable image. 3) From the TIFs, select images for posting, “Open Source…” (ref #1) them to resize and maybe crop and rotate, and save as JPEG. The TIFs are the result of my culling process, and also contain the commands I can use later to do “better” processing of the NEF with GIMP or somesuch (now a contender, thank you version 2.9!)
So, back to the original question, I really don’t see a problem with a plethora of alternatives. Indeed, this is what I find compelling about open-source software; the availability of alternatives that contain the thinking of creative individuals. In my case, it may come to pass that not one other person ends up using rawproc (okay, stupid name, I’m not a marketing person), but at least I met my need, learned a bunch, and maybe gave others pause to think about different ways of doing things.
Thanks for listening(reading, whatever…)
g
2 Likes
Hi, and welcome!
A quick word about photoflow:
-
it is designed to work well on low-end machines. I’ve been able to process some D810 RAWs in a virtual machine with 2GB of RAM. The peak memory consumption was around 500MB
-
you can get the same look as the camera Jpegs almost out-of-the-box: http://photoflowblog.blogspot.fr/2014/09/tutorial-how-to-match-nikon-in-camera.html
-
you can save your typical processing as a preset, including the resizing for the web. The resizing tool is implemented as an adjustment layer, which can be hidden by default in your preset and only made visible when you decide to save a Jpeg for web output.
I can give you more details if you are interested.
Also, which issues did you encounter with erlier versions for Windows? I have to admit that I had not much feedback about the windows version so far, so I would be really interested in your experience…
Last but not least, I would be really interested to see your touch-friendly gui… which toolkit are you using? Could you notify me if one day you’ll put your code on github?
dt should support HDRMerge’s DNGs in general, just those Deflate compressed ones are not supported by rawspeed. Could you provide an uncompressed one?
Compression is not optional.
@CarVac what sort of tone mapping does Filmulator use? Do you have a link to a paper on it? And how does it handle haloing, is it edge-aware?
Carmelo,
Thanks for responding. I think my challenges had to do with an earlier version, win32 binary, and IIRC I was able to eventually install it logged in as administrator, in spite of having that permission in my user account. By that time I was lips-deep in coding my little project so I just spent a couple of minutes trying things out. I do have to say that, of all the software out there, photoflow comes the closest to supporting my needs, followed somewhat distantly by RawTherapee.
WRT touch toolkits, I don’t think we’ll see much there in the next year, Microsoft seems to be taking that opportunity to compel developers onto their platform exclusively, and the cross-platform toolkits are just beginning to think about how that would work, and will be challenged by Microsoft’s approach. So, for my project I needed a slider, so I wrote one. Has a really big button for sliding, and you don’t have to hit it precisely with your finger to get it going. One of my to-dos prior to posting is to clean it up a bit, but I may put that off in favor of some stability debugging.
I would like to take the opportunity here to put in a small plug for the wxWidgets cross-platform toolkit. I’ve been using it for a while on Windows, and this recent project was my first cross-platform experience with it. Took a little bit of study to get it compiling in both Windows with Mingw and Ubuntu with GCC, but now I can pretty much do major changes in either OS and they just work in the other one. To get to touch soonest, Qt may be the best alternative, but I’m going to give it a go with wxWidgets first, even if I have to write more of my own widgets.
1 Like