PhotoFlow vs Filmulator

There are several courses of action here for Filmulator.

The first one is something I’ll do regardless: I’ll catch the crashes and not accept floating point DNG files, and instead simply offer stacking and bracket blending within Filmulator to achieve the same purpose.

The next is to accept high dynamic range non raw formats like EXR and a logarithmic integer tiff and maybe floating point tiff as well, and also have them be available as outputs from the initial loading stage for stitching. I’m hesitant but at the same time likely to do this. It’ll be effective and easier than completing built in stitching, but in terms of the program structure, inelegant and kludgy.

The last is to switch raw loading libraries. This might take some time, because right now I rely on LibRaw to perform demosaicing. My Halide implementation of LMMSE isn’t finished (it gives valid output but needs optimization), and I need to figure out highlight recovery, but once I do, that will give a large boost in performance especially when we get gpu acceleration working.

@Carmelo_DrRaw as a user any method is fine as long as it works :slight_smile:

@CarVac I don’t understand your stance - willing to support messy log and fp TIFFs which could be anything while not wanting to support fp DNG covered by the DNG Version 1.4 specification. But then you did write, [quote=“CarVac, post:4, topic:688”]
Filmulator is first and foremost for me to use.
[/quote]

@Morgan_Hardwood
Talking about HDR and floating-point formats, I am currently working on an experimental branch that not only supports reading and writing unclamped floating-point TIFFs, but also implements unclamped (i.e. HDR) editing for several tools (for example you can apply a negative exposure compensation to a floating point TIFF image to recover the details that were originally >1.0).

There I plan to include EXR and HDR DNG support as well (at least for reading).

This seems to be particularly interesting for scene-referred editing in the VFX industry, but probably it can be significantly useful for more “normal” photographers as well…

Sure! I’ll try to prepare some nice screenshot to show how the interface looks like, and few sentences to summarise the main features. This independently of a possible dedicated talk.

Thanks!

It’s a matter of code cleanliness and performance consistency. I don’t want cascades of different raw handling libraries that won’t necessarily give equivalent results.

You’re right about hacked log tiff being a bad idea though, I’ll completely scratch that.

The whole file format situation we’re in is appalling. As with many things in life, you can make a triangle out of this situation. In this case, in the corners you have: image storage options, metadata support, and licence situation. You can have a great format like BPG which supports various storage options (lossy, lossless, various bit depths, etc), great metadata support, but you can’t use it because patents. You can have OpenEXR, no licencing issues, good image storage options, but doesn’t support Exif, IPTC nor XMP (as far as I know - please prove me wrong!). Then there’s a bunch of formats with free licences and support for metadata, but image storing options from the dark ages. Ugh. There’s a bunch of good technologies which aren’t being widely used because some company demands exorbitant licencing and royalty fees, so they’d rather sink and take our generation down with them instead of facing facts and releasing the technology openly because they won’t make a dime off of it anyway. Oof.

Nice thing about HDR DNG is that it has all the benefits of typical DNG - great metadata support, various storage options, no licencing issues I know of, the files are very small - a 16-bit floating point HDR DNG made from five 12MB raw photos ends up taking about 14MB - so much space saved! And yes, 16-bit with floating-point precision is all you need.

But why should I need to deal with HDR DNG at all, unless cameras start outputting that natively?

Filmulator will support merging within the application, so the only benefit of HDR DNG is that the second time I work on it it starts up a bit faster.

I’m not going to delete the raw files, so there’s no space saved by saving any merged version aside from the final output.

  1. What justifies their existence considering what’s already on the libre graphics market?

Eow, here goes my first post, in a topic for which I have a certain interest…

I’ve spent the last year after resurrecting my photography hobby (into the digital age, what a head-hurter…) trying the various raw processors and generally being less than fully satisfied, 'cept for two: 1) dcraw, just does one thing and does it well, and 2) my in-camera firmware that produces JPEGs that look okay enough to which I don’t feel compelled to do too many subsequent manipulations. For feedback to the participants, I tried photoflow but my primary computer is a Windows tablet and I struggled with getting the earlier releases to run; until now, have not heard of Filmulator but I’ll give it a look.

Anyway, I’ve developed my own sense of what to do with my photos, and none of the tools did it all simply, so I started writing my own. Yes, yet another one, but I’m not that concerned about adding it to the ‘market’, so to speak. I’m probably going to put it on github in the next couple of weeks, and the only mode of support tendered will be gitub/fork. It uses FreeImage as its base, and responds to the following needs I couldn’t meet with any other software:

  1. I hate sidecar files. So, my software saves the modification commands in the UserComment exif tag, like this:
    rawproc-0.1 DSG_3890.NEF gamma:2.2 curve:16.0,0.0,64.0,109.0,131.0,194.0,220.0,256.0 saturation:17
    It also has a “Open source…” menu selection that will find this tag, open the source file and preload the processing commands.

  2. I wanted one tool to quickly get from .NEF to internet-ready .JPG. Funny, thing, I found the processing that hardest to do in this regard is resize. After doing just a few things to my 16-bit image, I simply want to make 640xXXX images for posting, and that last operation is either 1) non-existent, or 2) full of parameters from which to choose.

  3. I wanted a tool that would work on my cheap 32-bit Windows tablet. I bought a Microcenter Winbook tablet to read when I travel, and for photography to do what I call ‘contact sheet’ processing in the field. Most of the GUI applications are really to ‘heavy’ to run on a 2G 32-bit machine, and the UIs aren’t touch-friendly. Ironically, I found the Windows 8/10 Photos application is really pretty handy in this use, but I’ve only edited my camera JPEGs with it and I’d rather move to something I know is working in the 16-bit range. Writing touch-friendly software is a bit of a challenge right now, having to make my own sliders and toolbars, but it keeps me out of the bars…

  4. I don’t want image management. In that regard, Photoflow was the closest thing to scratching that itch. I’ll use the OS file explorer to manage my images, thank you. And, referring back to #3, I found that software that included image management was a challenge to run on my cheap hardware. I envision my field workflow to go something like this: 1) copy my NEFs to a directory. 2) scan the thumbnails for the good ones, and make TIFs from them with basic operations to render a usable image. 3) From the TIFs, select images for posting, “Open Source…” (ref #1) them to resize and maybe crop and rotate, and save as JPEG. The TIFs are the result of my culling process, and also contain the commands I can use later to do “better” processing of the NEF with GIMP or somesuch (now a contender, thank you version 2.9!)

So, back to the original question, I really don’t see a problem with a plethora of alternatives. Indeed, this is what I find compelling about open-source software; the availability of alternatives that contain the thinking of creative individuals. In my case, it may come to pass that not one other person ends up using rawproc (okay, stupid name, I’m not a marketing person), but at least I met my need, learned a bunch, and maybe gave others pause to think about different ways of doing things.

Thanks for listening(reading, whatever…)
g

2 Likes

Hi, and welcome!

A quick word about photoflow:

  • it is designed to work well on low-end machines. I’ve been able to process some D810 RAWs in a virtual machine with 2GB of RAM. The peak memory consumption was around 500MB

  • you can get the same look as the camera Jpegs almost out-of-the-box: http://photoflowblog.blogspot.fr/2014/09/tutorial-how-to-match-nikon-in-camera.html

  • you can save your typical processing as a preset, including the resizing for the web. The resizing tool is implemented as an adjustment layer, which can be hidden by default in your preset and only made visible when you decide to save a Jpeg for web output.

I can give you more details if you are interested.

Also, which issues did you encounter with erlier versions for Windows? I have to admit that I had not much feedback about the windows version so far, so I would be really interested in your experience…

Last but not least, I would be really interested to see your touch-friendly gui… which toolkit are you using? Could you notify me if one day you’ll put your code on github?

dt should support HDRMerge’s DNGs in general, just those Deflate compressed ones are not supported by rawspeed. Could you provide an uncompressed one?

Compression is not optional.

@CarVac what sort of tone mapping does Filmulator use? Do you have a link to a paper on it? And how does it handle haloing, is it edge-aware?

Carmelo,

Thanks for responding. I think my challenges had to do with an earlier version, win32 binary, and IIRC I was able to eventually install it logged in as administrator, in spite of having that permission in my user account. By that time I was lips-deep in coding my little project so I just spent a couple of minutes trying things out. I do have to say that, of all the software out there, photoflow comes the closest to supporting my needs, followed somewhat distantly by RawTherapee.

WRT touch toolkits, I don’t think we’ll see much there in the next year, Microsoft seems to be taking that opportunity to compel developers onto their platform exclusively, and the cross-platform toolkits are just beginning to think about how that would work, and will be challenged by Microsoft’s approach. So, for my project I needed a slider, so I wrote one. Has a really big button for sliding, and you don’t have to hit it precisely with your finger to get it going. One of my to-dos prior to posting is to clean it up a bit, but I may put that off in favor of some stability debugging.

I would like to take the opportunity here to put in a small plug for the wxWidgets cross-platform toolkit. I’ve been using it for a while on Windows, and this recent project was my first cross-platform experience with it. Took a little bit of study to get it compiling in both Windows with Mingw and Ubuntu with GCC, but now I can pretty much do major changes in either OS and they just work in the other one. To get to touch soonest, Qt may be the best alternative, but I’m going to give it a go with wxWidgets first, even if I have to write more of my own widgets.

1 Like

Filmulator is a “dumb” algorithm: it is a simulation of film development, no more, no less.
No edge awareness, nonlinear, and even at zero “strength” it inherently affects colors due to channels interacting. It affects colors more in smaller color spaces and less in bigger color spaces.

Film consists of tiny grains of silver in a silver halide emulsion. I assume they are infinitesimal and evenly distributed among pixels. A certain number are “exposed” by the raw file, according to a tone curve that is mostly linear but rolls off in the highlights. (Ones that aren’t exposed are ignored from here out)

The exposed grains then grow, consuming silver halide and the developer immediately above each pixel. Developer diffuses between the pixels and between the pixels and a bulk “reservoir”. Once (or not at all) all of the developer is “agitated”, completely evening out all the concentrations.

At the end, the cross section of the grains of silver are inverted and tone curved to give the final image. We do all the channels with their own silver halide concentration, but they share the developer, so bright channels will steal from the other channels at the same pixel location, intensifying bright colors near the highlights.

They also steal from neighboring pixels, thanks to the diffusion effects. This leads to halos which may bother the allergic (@Ofnuts) but I am of the belief that subtle halos of the correct size for the display format will be imperceptible for most viewers, because they’ll mimic the halos that the eye itself exhibits when observing a scene, thereby implying the dynamic range of the original scene. Edge aware algorithms I’ve seen don’t evoke the original brightness range and so look unnatural to me.

1 Like

That’s an original approach!

I was surprised nobody had done it before, in pursuit of the “film look”.

1 Like

CarVac - I checked out your demo images and was impressed. I really like what you have achieved, and think the approach is brilliant. In the near future I would like to give Filmulator a try.

Will you make builds available in the future, or do you expect that it will always be build-your-own?

Thanks,

Eric

I’d love to make builds available, but I’m a noob at packaging so I could use help for that… I don’t even know how to deploy the build.

Same with porting: I wouldn’t mind making it available for Mac and Windows, but I have neither type of computer to develop on and I need to know filesystem placement conventions for the database and thumbnails and stuff (that should be the only barrier to porting it at the moment).

I just recently set up my W10 machine with Ubuntu on the side. I am slowly getting ready to get back to some photography, and have been impressed by the energy around the tools here.

I am guessing you are working in Linux? I’m a lapsed developer, and rusty at building, and so far don;t know how to use GitHub. My quick scan of GitHub looked like the modules were kind of scattered. Has anyone else built a copy of Filmulator.

It looks like the new version of LibRaw (0.18) supports Floating Point DNG, see here.