Scene referred, display referred, and the Colour Science of Canon/Fujitsu/Nikon/etc. in Darktable (filmic-rgb module in particular)

I personally like my starting point to be as bare bones as possible, so: Neither of the above. That being said…

If I do have to choose a starting point and the above four are my choices: Number 4.

Posted by @srgmro

1 Like

Thats an interesting blind test. A couple of different scenes would make it very useful for devs?

I think there is a tendency to forget that huge amounts of effort has gone into “looks” over the years (including film) and that those are cultural not scientific. Our brains read images with the images we’ve seen before as reference. Cooking this from scratch isn’t really feasible for almost any user. Our software always makes some decisions for us but where to draw the line and what look is default is a huge part and draw of our tools.

It’s not a democracy, it’s a docraty. Things happen because people make them happen, at the end the decision is up to the ones doing the job because that’s the only reward they will get anyway.

Ideas don’t solve problems, they are free and anybody can have them. They don’t even have to be relevant or realistic. Ideas are worthless until they become products. Different software cater to different needs and motivations, the only tool every photo editing app should have is gamut mapping. Trying to please all users at once is only going to displease them all, some of them have irreconciliable needs and expectations, pleasing all sides is going to end up in a convoluted software with too many modes that will be even more confusing.

darktable has an edge on any other software : the ability to affect chromaticity and luminance fully independently and advanced masking options capable of handling HDR with minimal distortion of hues. That gives a lot more freedom than any other soft around. But, yes, freedom is overwhelming when you are used to the leash.

2 Likes

3 and 4 have washed sky degrading to cyan and close to no details in the clouds. That’s completely unacceptable.

1, 3, and 4 are over-sharpened.

As a working base, I prefer 2 for its neutrality and lack of opinion.

4 is definitely the Adobe look. I bet 3 is the OOC JPEC, 1 is filmulator, and 2 is darktable.

3 Likes

I will make one final - last ditch attempt to get all of us to see through the haze of sticking to a legacy approach, that does not work for everyone, and I am not alone in saying that we can do better, I am sure there are other voices amongst the darktable user community who share a similar opinion, but may not be as vocal.

  1. The purpose of a tool, is to make life easier. Not more difficult.

  2. Any additional functionality introduced into darktable will not take anything away from what is already there. Neither does it discredit any of the hardwork that it has taken to bring darktable to where it is today. Change and progress - that is life, and I would have hoped that a community of open source people and open source users, will also be a lot more open minded. I must be frank, the conversation and conservatism in this group from some has been dissappointing. there is a bigger world out there, which darktable can tap into for ideas to become a better tool. darktable’s origins have already borrowed lots of ideas from existing tools like Adobe Lightroom and Photoshop. What’s the problem with borrowing a few more good ideas. None.

  3. I have read the conservative clamour for keeping things the way they are, from those who like darktable as it is, no issue, adding new functionality does not take away the legacy you like, just do not clamp down on new ideas that may improve darktable for others like me (and I am sure many others).

  4. Aurelien and others speak of a cherished retention of providing “controls” so users can tailor to their hearts content in their own individual way. Implementing new functionality in darktable in a new module, will not take away the control they already have.

  5. If you take a proper step back and look at darktable, you find right in the filmic module, and in many other modules, a button or icon, labelled “Auto”. Interestingly the english version of the darktable 3.4 manual mentions the word auto either on its own or in words like automatically - 146 times. That’s what darktable also does, automatically doing so many things for the users, and some of it can be by their choice.

In the filmic module - you have the auto button, so all this clamour about user choice is not taken away, by adding an auto feature. You can have darktable do certain things automatically by a user chosen choice to achieve something. In the case of the filmic module it does some analysis, obviously and then uses its in-built “Colour Science” to make a guess at ideal settings. Other modules in darktable have similar features on buttons or other controls like eye dropper tools, to make things easier. This does not in any way take away from the control features available. Within the filmic module, if you use the Auto function, you still have the ability to go ahead and improve or alter the automatically chosen settings.

So its not one or the other, you can have control and also have assisted “photo intelligence” in the same tools, or modules and the end user who should be the most important customer of the tool has a choice, but also has the option to take advantage of any assistance provided in the tootl - an option - you do not have to use it.

The way it has been managed in filmic is great, you have direct access to the settings automatically achieved by the auto button, andcan still change the results, to what you prefer.

I would suggest the new functionality that provides a ready to use image, be created in another module, so that filmic module development can remain independent according to whatever Aurelien had intended for it.

I suggest that the new functionality to address the concerns I have raised for a more end user friendly image as a starting point in darktable, be implemented as a module somewhat similar in approach to filmic where all the parameters that the tool uses to make an accurate “guess” of the best settings, are available as sliders, buttons, or whatever is the most suitable way of presenting this in the user interface, and all we need is an auto button, which when clicked results in the module which I will call “Polaroid” for want of a better name - but you get the idea - simple one click, and you get a nice image, but all the controls and sliders have been adjusted so you still have control, and much more control, cos you can go ahead and adjust to your hearts content.

Control is not taken away from you, actually it is enhanced, you have direct access to the intelligence behind such wonderful photo “colour science” that can inteligently analyse an image and adjust settings to present you a wonderful starting point. You do not have to use the auto button in this new “Polaroid” module, but its there for those of us who like to use it, same as you find in the filmic module. One click, you get a great image, if that is what you want, and you can then tweak the settings and use the other modules in darktable to complete your creative intention.

I reiterate, we have one good example of this is an open source tool - Filmulator, and if I could code myself, I would do this.

Maybe I need to challenge myself andt learn to code in C or C++ like Aurelien had to also learn, a few years ago, and like him, add this new module to darktable. It would be so much faster though if others who have far more abilities than me, saw the value in this and could achieve this much faster and more completely than I could.

The key 1st step would be to understand the code in Fimlulator and decide which variables are set by the “embedded” photo intelligence and how to expose these variables as controls, so that when the auto function in the new “Polaroid” module is activated, this sets the values of the exposed controls. Of course there could be default settings like in filmic. So you have three options, stick with the default generic settings, click on the Auto button to get enhanced settings of the controls, or start from manual and tweak the defaults to your liking without any assistance by and Auto “photo intelligence - colour science assistant” button. Pretty much like it already is in filmic, just more like what we have in filmulator, with the potential of instant high quality as a bonus.

Certain frustrations led to the development of filmic, and I’m making a genuine case for more. maybe some of the existing or other new developers will also catch my frustration and help out, to add much needed functionaility to darktable.

At the end of the day what distinguishes humanity is our persistent development of tools, and it does not matter who makes them, what’s important is that we move forward. Each of us can choose to play a part in progress or decide not to, its our choice. I would sincerely hope that we can move darktable forward. If I can think of these ideas, I am sure many others are likely to have thought similar thoughts, I only happen to be the one at this time kicking up a fuss about it cos I’ve seen this implemented, and moreso in an open source tool. We can do this and close some much needed gaps, in darktable.

It’s ony a matter of time, someone will add this feature to darktable - to provide instant hight quality starting points, similar to what is achievable in other tools Lightroom, Photoshop, Filmulator, and on Mac you have RAW photo processor which is similar - freware, but closed source(to the best of my knowlege), and we can either participate and contribute to making this happen.

Base curve - great starting point.
Filmic - even better.
“Polaroid” - next step. which will happen one day, its only a matter of time. “Polaroid” will combine the simplicity of Base curve, with some of the kind of user interface of Filmic, to combine simplicity and control, so you have the best of both worlds and above all a high quality auto finished product of a pretty decent image, if you so choose to click that “Auto” button, and still have all the control to modify the auto generated settings.

It will happen, its only a matter of time. Someone will do this.

1 Like

@OK1 I’m sorry, but despite your eloquence and visions of utopia, you are deadlocking any sort of reasonable discussion. You insist darktable misses the point and is conservative in its approach, but your only argument for this is that the starting point for editing is sub-optimal. You want presets, simple, automagical solutions. People try to explain to you why that is neither their intent nor desirable in general. However, you don’t engage further to understand them and seem as closed-minded about this situation as you claim the others are…

Case in point: you claim Filmulator and other software can prove that things can be better in darktable. Yet, when I provide a blind experiment to test your claims, you (so far) refuse to comment and instead write another 1448 word post rehashing your prior statements. In other contexts this would be considered trolling… :man_shrugging:

15 Likes

You are asking for something that can’t be defined. A ready to use image for most professional colorists esp in video is as flat as a pancake. That is where they would like to begin with a mostly blank canvas with the most room to move. Others like yourself may want it more like what they expect the camera to produce. If that is a huge priority then the manufacturers software is the best starting point without all this back and forth. Open it there save a tiff and edit the rest elsewhere or in DT as you see fit or cobble together the tools you want.

You have written hundreds of words without any concrete examples or comparisons so for now all one can conclude is you like the look “you” can get in filmulator and not what you can get in DT and you believe it to be easier.

I believe in another post @CarVac pointed out why currently Filmulator as it functions now would not be compatible with the pipeline used by DT so you keep asking for this module but its not that easy to just introduce it.

You seem to have a vision and if it is that important then I am sure you will find the time and acquire the knowledge to fulfil it and if not you rely on the tools at hand.

2 Likes

I’m almost shocked by that reply :grinning: Number 2 has gigantic swaths of lost detail and unrecoverable highlights, while the clouds can easily be salvaged in number 4. Degrading to cyan? That would be nitpicking in this context. :slightly_smiling_face:

Number 4… Just a simple curve pull will give this:

Number 2 gives me this:

This is not edited for pleasing aesthetics, but rather to show what’s there to work with (after being saved to 8 bit JPEG, of course).

I agree that number 4 is ACR and that number 3 is probably OOC JPEG. I have no experience with Filmulator, so I’ll leave out the rest of the guesses.

My two cents, as a plain user, much like what @OK1 seems to be.
What you @OK1 don’t seem to understand is what lies behind any of these FOSS software: a community gathered around some interests, or values, that end up being instantiated in a product like darktable.
It’s important that you understand that what you’re really trying to do isn’t change the way the product should be (by means of ideas), but actually trying to change the community’s values, in this case, the darktable community.
I’m not sure if you realize that this is a bit pretentious - like someone who’s invited to a home and start saying how people should be and behave.
I don’t agree with any of the items you outlined above because none of them consider the community and the way it works. Those items clearly describe a commercial vision (making life easier, democracy, users, …).
As people said before, software like this starts from someone trying to produce something the way he/she wants, then gathering more people around the idea, and in the long run, of course there will be changes, but they probably will stick inside the original idea and vision. If a change needs to replace that vision, it’s a clear sign that another community is needed. And that’s the beauty of FOSS: you can fork it.
EDIT: Not to mention that I find it a bit disrespectful to Filmulator’s community: you seem to find it useful (means, it fits your vision), but you still insist in changing darktable. Why don’t you embrace Filmulator once for all and add value to its community, by start editing and contributing with compliments, bug reports, tutorials?

2 Likes

Making literally one change in the GUI for Number 2 gives me this :wink:

For Number 4, also one change in the GUI:

1 Like

Hooray, that looks much better! :smiley: But I assume that change was made from raw data. All I had to make judgements from was a JPEG with lost details.

Filmulator doesn’t work well in darktable says filmulator’s creator: Filmulator v0.11.0 released! - #26 by CarVac

We are lucky that we have a bunch of different applications that achieve different things. Smushing them all together means we loose that nuance.

1 Like

Isn’t it mostly just a matter of taste and time?
Personally I hate the OOC output of my cameras in about 90% of the cases. But if things have to go quick (n dirty) I have to confess - yes I sometimes simply took the OOC Jpegs and adjusted them a bit with gthumb and done.
Maybe I will take the time to optimize my cameras features and get more out of camera, but if not dt is always there as long as there is a raw.
In most cases I use the OOC Jpegs only to sort out unsharp or completely unusable images and then open my new shot raw images and enjoy all the endless options to mold it into either how I saw, felt or remember the scene, colors, contrast etc. or to create an impressive desirable look. An automatic OOC option for darktable may be desireable for some users - then maybe just open your jpegs with darktable to do the finetuning :wink:
The main reason to grab my dslm over the iphone is raw and darktable. When it comes to OOC images the phone is at least as good as the dslm, often better.

While I obviously prefer Filmulator’s results myself, there are obviously many who will not. Either you like it or you don’t.

In the dpreview comments many people didn’t see what the big fuss was about—and that’s okay. They don’t have to use it.

But it’s not self-evident that Filmulator is better. You cannot just say “see for yourself” and expect everyone to agree.

There are two confounding factors here, though, given that you’re talking to the darktable community.

One is that people get used to the way their tools render images, so hardcore darktable users (and devs) are not going to be inclined to see other editors’ results as better. I look at others’ people’s edits and sometimes think to myself “Filmulator would do that better”, and the people here might say the opposite.

The other is that the people who use darktable are the people who already liked darktable’s output. People like me who do think Filmulator’s default output is better already don’t use darktable.

7 Likes

Well the auto (based on what?) button is something that needs more discussion.

However i think darktable has got many good features over the last ten years, for me they are:

  1. Amaze demosaicing
  2. a good wavelet chroma denoiser
  3. a good luma denoiser (I prefer the astrophoto denoiser)
  4. rgb curves and rgb levels (with preserve chrominance set to no)
  5. lch equalizer

There are something very basic missing like:

  1. a not bugged highlights recovery module

  2. a channel mixer that doesn’t produce ton of artifacts (the colorfulness and brightness sliders shouldn’t be there because they’re not based on how channel mixer works), the gamut compression slider is mostly useless

  3. support for DCP profiles and dual illuminant matrices .

  4. something like the “rawtherapee’s avoid color shift” after a lab adjustment Lab Adjustments - RawPedia

1 Like

I think it’s easy to create an auto-optimization in darktable. just create 3-4 module-presets and make them be applied automatically to your photos. In addition to exposure and filmic that would be local contrast, chromatic aberrations and contrast equalizer for sharpening, maybe denoise. Then you get the same results as in Lightroom and C1. It’s really no big deal, it’s almost the same for every camera and there is no big science behind it. You probably even get better results with such an approach than in C1 and Lr. I guess it would not be very difficult to add an option in the dt settings “automatically apply style xy” or “enable auto-opimisation”, but the question is, is it desirable.
What I don’t like about commercial raw developers is the fact that it is impossible or difficult to turn off this kind of auto-optimisation.

3 Likes

This. It’s the only reason I’ve more or settled RawTherapee as my LR replacement. Not being a developer I wonder: would there be any chance of using RawTherapee code for highlight recovery in darktable? This was also a very interesting post from the other day.

DCP profiles would be an added bonus.

Bingo! What is the point of trying to use advanced tools to try to accomplish what you already have to start with?

1 Like

Playing devil’s advocate, I might reply: the point is for sophisticated software to set its controls such that the OOC JPEG is re-created. Then, if I want, I can adjust any of the controls to improve the image.

I think that objective is (a) infeasible, given the huge variety of cameras and their algorithms, and (b) undesirable because I think the OOC JPEG is a poor starting place.

7 Likes

Do we assume for this hypothetical question that we “like the rendering of a camera” and you just ask why one would still use a raw developing program instead of OOC jpegs? Then there’s a whole slew of reasons I think:

  • Higher bitdepth in TIFFs for example (for printing, the printer-driver might like this)
  • Better demosaicing than OOC
  • Better denoisers than OOC
  • Better hot-pixel rejection than OOC
  • Better capture sharpening than OOC
  • Better CA mitigation/ color-fringe mitigation

I also find it a bit problematic to generalize to OOC jpegs in general (that’s a rather new thing I read on here). As if there is one OOC jpeg recipe, Fujis jpegs are on the better end of the spectrum, canons are mostly garbage. The frowning-upon-OOC-jpegs is frequently pulled out as a straw man I think, as it’s of course is easy to look down on.