Scene referred, display referred, and the Colour Science of Canon/Fujitsu/Nikon/etc. in Darktable (filmic-rgb module in particular)

Long story short. Sadly, going forward, I can only use Darktable for special cases, where I’d like to deviate pretty creatively, from the out of the camera jpg.

For about 10 months, all my edits have been in darktable, and done my best to learn it and produce great results with it, but in some cases, like a portrait photo I took recently, it was such a herculean effort with darktable.

My big mistake was after several weeks of editing this set of photos, in darktable, some which turned out pretty nice, I made four inadvertent comparisons.

  1. Took a look at the out of camera jpg, which is embedded in the raw file (you can extract this using a variety of tools downloadable from the net)

  2. Tried editing a photo using Sony’s accompanying editing tool. I have owned a Sony for a few months but never bothered to use their editor, as I had stuck to darktable. One thing was pretty clear, the Sony editor was as identical in results (if I did not change any parameters) as the out of camera jpg, as I can imagine.

  3. Installed Adobe Photoshop Express, which is where I started a bit over a year ago, and moved from this to darktable, and the results I was getting with Adobe Photoshop Express were quite close, especially in the level of “detail” that was retained in teh sharpening of the image, when compared with all the sharpening options I have exhaustively tried in darktable - highpass, contrast eq, sharpen modules. Pretty similar and retaining same level of detail as with the Sony Editor - Imaging Edge.

  4. Installed the Capture One version for Sony, and while being a more difficult tool to use, and learn, than the aforementioned, the tool seemed more “photographic” in its end results.

Without directly comparing with other tools, which I never really bothered to do until I had exhausted all my options in darktable, darktable is a commendable effort.

But IMHO, when I have compared the end results, losing much sleep with darktable over the last month or two, unfortunately results which I cannot share publicly as these are photographs of people, I have to admit, like it or not, there is a huge amount of “inteligence” that has gone into the tools that are not open source or freeware like darktable, i,e the free tools from Adobe(Photoshop Express), Capture One, and Sony, which make a compelling case, for most people to use these, instead of darktable, for a lot of their images.

I still think that there is tremendous value in shooting raw(and moreso raw + explicit jpeg generation - as distinct exportable files - in camera), especially with the knowledge that if ever there were improvements in color science, or image processing in the future, much like we have remastered music, these raw files are our own negatives from which we could develop even better or more fashionable or pleasing results, whenever technology catches up with the abilities of human vision.

We could imagine future where display technologies and maybe printing technologies have advanced to show us more colours and support higher bit depth output, which is already taking place with HDR, 10 bit and who knows one day, we’ll have much more capability on the output end of things. In audio 16 bit was de-facto for distribution and end-user consumption, but things have moved on and now we can have 24 bit high res remasters and remixes (in theory and in practice). Same should apply to video and photos, in the future.

I am aware that somethin akin occurs in the Blackmagic camera world where improvements in their colour science allows you to reprocess pseudo raw video files(cos Blackmagic capture is not really 100% raw) , using an improved proprietary version of their process.

Learning darktable has not been a waste of time, not so. Still a tool I would use on special occasions. The ecosystem and contributors to darkable have opened up knowledge that I had no clue about, such as the current limitations in digital photography, needing to conform images to colour spaces from input to output, needing to reduce the bit depth of images, becoming aware of out of gamut challenges that othr tools never show you, in their effort to keep complexity away from you. So much has been learned by trying to get darktable to deliver the results I wanted.

Over time as my picture taking has improved, and my vision, of what I want to achieve has also improved, I cannot ignore the pretty significant amount of “intelligence” that has been embedded in the proprietary image editors. Straight out of opening up the RAW image, you get a more believable starting point, and while I accept that this starting point is a bit of a straight jacket, roping you into a samey samey image, but its also a safety net. Darktable is like high wire tight rope walking, with very little to stop you from making a right mess of things, which is also good - you learn from the falls.

I do not think the “intelligence” baked into proprietary tools and cameras will ever be available in open source tools, unless some unforseen failure or government supported initiative made this neccesary, so for photo editing, there will be the two camps. Open source, and proprietary, and they from what I have extensively researched, will never look similar. Never. For certain kinds of images, I’ll go with darktable, for others, darktable will have to take a back seat, cos the initial image presented by darktable, for very good reasons, deviates quite a bit from the image presented to you as a starting point in any of the proprietary closed source tools. Thats a fact, not debatable.

But the images presented to you by the proprietary tools, looks much more similar to each other, across various tools from Adobe, Capture One, and directly from Sony.

Let’s be open about this, and not defensive. Darktable will struggle to incorporate this comfortable starting off point, when you open up your raw image. It cannot, unless something unforseen happens, invent this “photo” intelligence, that is locked up in Sony, Canon, Nikon, and Adobe. For quite a bit of the other image editing, sure Darktable can come in handy, once the initial image is developed elsewhere. I could imagine importing a 32 bit tiff into darktable, after inital “development” of photos in some other tool, in some use cases.

I know that these opinions will not be popular, after all this is a forum for open source tools. But I hope I can contribute something meaningful, by pointing out the shortcoming that I cannot see any open source tool overcoming, in the near future, unless something unforseen occurs, such as a kind donation by one of the big players, of their photo intelligence, to the open source community.

I can imagine that many in the free and open source community will challenge my opinions, but please do your own research - take the same images, and edit them in darktable, and in the proprietary tools, and take a good look at the results, especially the starting point of photo development in the tool. Its a lot harder in darktable cos there is something missing in the various start off points for photo development in darktable. I have tried all four approaches and they produce inconsistent results across various kinds of photos.

The four approaches, being to conform the raw file using as a starting point.

  1. Using rgb curves or tone curves to add the required contrast, such as a 2.2 gamma.

  2. Using LUTS.

  3. Using base curves

  4. Using filmic.

A lot of the time you get a decent enough result, and “it depends” but you never really know what image is better processed with which initial approach until you try them all, which is a pretty huge overhead to the workflow, only if you have lots of time. Creative - yes, fun yes, efficient - no.

No knowledge is lost or wasted.

My final take is - as my pictures have improved, in the taking of them, there has been less and less of a need to edit. Getting the lighting right, framing, avoiding poor exposure choices, and with all these advances, less and less of a need for extensive edits in darktable, but more of a great need to just have the image pop up properly on first open, as it does on other tools, not having to start this hurdle or massaging the image, to look great from day 1, and cycle through a few of the initial photo development paths open to you in darktable (listed above).

Today, an image I had struggled with in darktable, creating several versions, which is darktable’s strength - great image processing approach, in theory, of which I liked only one or two versions, after many revisions. Then I opened up the same raw file in Adobe Photoshop Express and in less than 20 seconds, just a little bit of extra sharpening, that was it - done. The raw file, straight out of camera as “developed” in Adobe Photoshop Express was 95% done, and I cannot fight with the obvious. Compared the results with all I had achieved in darktable, and no contest. It will take an unforseen event for open source tools to have that kind of “photo intelligence”. Its not just about darktable, its the same with Raw Therapee - the research investment in “photo intelligence”, in comparison to proprietary tools is going to be pretty hard to match. Any non proprietary tool for developing images from the raw file, will face a similar challenge.

2 Likes