New Sigmoid Scene to Display mapping

Really like it from a first test ! To be honest i lost a little bit of enthusiasm with filmic and Color calibration etc in the latest versions. I know they are very good technically, but when the defaults don’t work it feels like work to make the image I want.
This module gives this nice simple feeling to start with. It’s good to know that there are all possibilities to fine tune an image, but your module brought back the fun that was lost with dt 3.2 onwards.

1 Like

You know ART ? It’s probably better suited for you.

And don’t get offended by Aurélien’s proposal. We (the devs) have made a lot of efforts those last years to propose easier and more polished modules and UI. The introduction of a “basic module” and in 3.6 you’ll have a basic group where the simple controls for most dev work will be present. The layout of the modules have been simplified and come with ready-to-use presets. All this is documented of course and many hints have been added. If filmic & co are too complex for you we’ve kept the old base curve and dartkable propose a workflows based on it.

But darktable won’t go the route of one button to do the job. I’m not saying that’s bad goal, I’m just saying that it is not darktable. And so, if you’re looking for something very simple, just some buttons and happy with playing with modules with almost no control and where the rendering/algorithm is hard coded then you don’t want to use darktable.

11 Likes

I think filmic in v3.4 is the version easiest to use, I struggled a bit with previous versions. I made the effort to watch and read various tutorials to try and understand what filmic is trying to do and it now makes total sense. I can now get good images very quickly with little messing around. The auto tuners can do a good job if there are true whites and blacks in the image.

I’ve not tried colour calibration yet but this is next on my list of learning.

1 Like

@Pascal_Obry @anon41087856

Yes, i know ART and like it. When i boot into Manjaro i sometimes use it, but on Windows (which i also use for work) it just feels a bit slow on my PC.
I hope i did not sound offensive or rude. I really like Darktable. Besides being sometimes a bit slow UI wise (as almost all GTK Apps on Windows) it’s fast enough and the quality of DT is outstanding. All in all it’s my goto application for RAW development.

Having studied CS with ComputerGraphics and Pattern Recognition as my major subjects i know how difficult color can be and i also understand the problems DT has to solve.

Maybe it’s just my Corona Lockdown Winter Blues at the moment which leaves me frustrated with everything and photography at the moment. I will hopefully adept my workflow when times get better :).

For now i just use some filmic, crop my images, use NR and adjust some colors. And when things don’t work out i just apply some good looking LUT with DT and retouch with Affinity.
Photography is just my hobby and form of being creative, so it does not matter if it will take some time.

@Pascal_Obry I am not looking for something simple, it just takes some time for me to adept. And i think, coming back to this topic, this simple sigmoid slider module worked in first tests really fine.
I really apprectiate the quality of filmic out of the box, i just struggle to make the picture look like i imagine. But as i said i am not in a hurry

Edit:

BTW, thanks for all the work on DT!

1 Like

Do these functions work in the 0-1 range?
In darktable is missing a simple rgb brightness/contrast module for operations after the filmic module or for fast editing of ldr images, so this new module is very welcome.

I think that the user should have the possibility to change the middle gray pivot tough.

Is it worth to add the imagemagick sigmoidal contrast too?

https://legacy.imagemagick.org/Usage/color_mods/#sigmoidal

3 Likes

I think for some reason still the provided presets are not used by many users. In fact some seem surprised they exist when you mention it. I rarely need to use shadows and highlights or other tools for shadows. I hit the preset for relight in the tone eq and tweak a bit sometimes the exposure compensation and most often at least for a quick bulk edit this is fantastic. So no moving around the screen altering regions or tweaking the mask etc etc…I would extend this to many of the other presets in the other modules…they can be great starting points. I would say I have a collection of 20 or 30 presets scattered across the modules that I will use on my edits and I return to them time and time again…so you can certainly make life easier for yourself if you learn a bit and spend a bit of time.

1 Like

Would that not be a role for color balance??

I love the new color cal module. But I think I probably don’t use it necessarily as intended. I started by trying to understand all the options and apply them and worry about color temperature etc and it really worked okay but I was not really sure of the advantages. Then I stumbled on to a simple workflow.
I noted that if I selected the CAT transform and the autopicker it would often do a nice job but sometimes it was overdone or not offer the look I wanted. The real winner for me was that sometimes when it determined an illuminant that was not a standard one in the list I noticed it would present a hue and chroma setting instead of temperature. This was golden because I also discovered that if you select custom even when the module finds a matching illuminant then you will get the hue and chroma sliders. The hue and chroma are determined based on your selection area so the average of the image at the start but you can choose any area. So initially the hue and chroma reflect the cast that is being removed so if your image is warm then it may be yellowish. If you simply reduce the chroma to 0 there is no correction now you can move it up and down and fine tune the correction. This may be a bit hard to describe or visualize but you can precisely fine tune the correction.

So I don’t even think about temp anymore I just go to strait to this custom setting and tweak it. This is quite powerful. Normally your best choice would be to select something neutral but you can essentially select any part of your image that has a wb issue and then dial it in and out nicely with the chroma slider. I find it works with skin too. You select the skin and the module corrects for the wb of it and then you can reduce the effect to bring back just the right amount of color temp into the skin.
So while the module has a ton of options I literally set it to do the Cat16 transform autopick a region and select custom and adjust the chroma…this gives a very nice image. I imagine you could leverage this nicely for creative effects as well.

I do also use the colorfulnes and brightness tabs as well. They are quite useful too.

7 Likes

I’m a standard user and I feel at ease using the current framework (scene-referred and now the color calibration thing - put this shallow way, to illustrate how standard I am)
What I understand from all highly technical discussions is that there is a clear direction where darktable is going to, and, so far, it’s working for me, usually with minimal effort. And for all references made in the discussions, the strategy seems to be on pair with current image development standards.
So, I trust Aurélien when he says that this module is a duplicate (regarding filmic).
Besides, the current set of minimal tools to get a decent rendering is working very well to me (all personal edits I do).
This is what I achieved with a rather minimal set up, just raised exposure, adjusted white and black level on filmic, then lowered the highlights with tone equalizer and tweaked color calibration just a bit.

And if I’m not satisfied with the sky, I can duplicate color calibration, tweak the area picker, and apply a gradient mask to eliminate the foreground:

All in all, although I respect the work you did on the sygmoid curve, which clearly shows excelent results, I don’t miss it.

5 Likes

Yes of course! Sorry that I didn’t do that straight away. Seems like I can’t edit the post anymore here they are:

Noctilucent clouds: DSCF2925.RAF.xmp (32.4 KB)
Amulree Kirk: IMG_5044.CR2.xmp (6.8 KB)
King Fisher: PC251438.ORF.xmp (11.8 KB)

@age no, it maps intensities in the range [0, infinity) to (0, 1). A very different operation than [0,1] → [0,1] operation and has to be done once and only once in the pipeline.

1 Like

Oh wow, it did spur quite a debate…

Maybe some more background about me and what my idea was behind this will help.

I see myself as a pretty advanced darktable user with a strong background in physics and mathematics. My photography interest is mostly in outdoor sports, dance, insects, and photogrammetry. My daytime work is about both signal processing and user interaction. My experience is that things generally become much much easier when things are kept linear and on a correct orthogonal basis.

I think that THE best thing about darktable is its modules! I will probably not be a basic adjustments user as that kind of kills the strength of darktable. That being said. My view of a good module is the following, assuming that math, transforms and other processing is done correctly:

  • Single responsibility
  • Settings are of reasonable dimensionality
  • Settings are as orthogonal as possible

Single responsibility is pretty given, you do not want your denoise module to have a global setting for image saturation, it should only suppress noise.
Anyone can learn what a one-dimensional function does, turning up the heat on the stove makes the water boil faster. Two is still ok, for example learning the relationship between throttle and gear when driving a manual transmission car. Three is harder, four is much harder, etc.
I think high-dimensional settings are pretty ok though as long as they fulfill the last point, orthogonality. The problem with poor orthogonality is probably best known in image editing when it comes to HSV adjustment directly on sRGB data. A change in hue will also change the perceived brightness and or saturation.

So to me, something similar to the sigmoid module does make a lot of sense. It has one task, compress scene-referred intensity to a displayable range. It does so in, what I hope, is a similar way as the well-established cinema-standard ACES does. Its setting is one dimensional and thus fast and easy to work with.
Color-preservation ala filmic would be easy to add. But I would love a reference to some non-darktable source about that for better understanding. I still haven’t found it used anywhere else despite looking for it. Hue and especially saturation will change when doing this but doesn’t it have to if we want the image to fit inside of a viewable range? The sigmoid could of course be made more complex by for example adding user-defined definitions for black level, grey point, and white level. It is after all just an algebraic function that can be modified with at least three degrees of freedom.

darktable offers multiple methods for adjusting saturation, why wouldn’t this be the case for the scene to display transform?

I feel like I also need to reiterate that my initial intention for this topic. Introducing myself as a possible new contributor to darktable. Please do keep that in mind and I will happily read any resources that you link me to!

14 Likes

ACES does not have a super-special way of generating their ODTs (output display transforms) though…in principle you could generate and use wonky/‘wrong’ ODTs if I understood this correctly. Gamut and tone mapping operations are not strictly defined in aces AFAIK. So what do you like about ACES in particular?

I am not quite clear from this paragraph what references you ask for? Hue constancy upon chromaticity changes? Chromaticity changes upon changeing contrast (when using certain color-models)?
The Safdar Paper about JzAzBz is always a good read:
Safdar et. al.
Has a bit of color theory in it and some graphs to understand why hue constancy is a bit tricky depending on what you want to do.

Again: Welcome! :smile_cat:

1 Like

Thank you @PhotoPhysicsGuy, good to get some friendly welcome as well!

I’m currently doing some reading and further research on the topic (including diving in to the Filmic source to understand what I missed). But lets answer your questions in the mean time:

Why I like Aces?
To begin with it has proven itself with a lot of people and productions which do aim at scene linear workflows as well as outputs to both SDR and HDR displays. There is usually a lot of good stuff to be learned from others so comparing darktable edits to that of a ACES workflow is in my mind a nice sanity check.

I do really like the way especially the highlights looks, with a smooth rollover to a desaturated white.

As for references on preserving colors. Thanks for a good link! Putting my head into it :slight_smile:
To clarify, I do understand the importance of orthogonal Luminance, Chroma and Hue to make it possible to edit just one of them at a time in a predictable way. Saw btw your link to the OKLab space, only valid for [0, 1], but still a good example of how important orthogonality is!

My intended question was more about the compression from scene to display referred and why one would want to preserve colors here. Assuming that constant color in this context means constant hue AND saturation. I can see the possibility to preserve hue as that is just an angle but it seems hard to not change the saturation. The highlight desaturation in ACES, which is my main reason for love, implies a change in saturation and thus a change of color.

3 Likes

Sure. Always a good thing to do things like an industry standard does it. Sanity checks are always a good idea. Except for when the industry standard might not think about stuff because they usually do not have to deal with.

That touches on what I wrote before. In a series of ACES threads over on http://community.acescentral.com/ you’ll find very recent discussions about how gamut remapping should be dealt with for generating ODTs. Highlight rolloff is not strictly gamut remapping, but both things are very much related (If you look at color-volume plots in XYZ space for display spaces you’ll find that the closer you come to maximum luminance the more gamut remapping one has to do. Similar for shadows.). Then again, ACES never had to think about this, because if an ACES-to-rec709-odt had odd colors compared to an ACES-to-rec2100-odt, a colorist would do a proper job of grading for that output. Nobody in that industry would want a one-click-solution, or a dynamic range slider which opens up the highlights for HDR. They’d just regrade it on a new reference monitor capable of displaying HDR.

Also there are some people who criticised ACES for the highlight rolloff as well. Specifically for doing things like chromaticity shifts…so it might be worth checking if what you like about ACES might be the result of a skilled colorist, or if you indeed like the standard ACES highlight rolloff.

Well ‘perceived hue’ unfortunately does a some weird stuff when mixing achromatic light to it (‘desaturating’), see https://en.wikipedia.org/wiki/Abney_effect

Filmic tries to do a smooth highlight rolloff which becomes more and more achromatic, but that is just before going into display-referred output-space. I am assuming he means in working-space. Whatever you do to the dynamic range compression, chromaticity should be preserved (but raises gamut remapping questions). If I understood him correct, that is.
Oh and another idea to watch out for is that in order to have things nice in display-referred [0,1] you need to know values of over 1 in your working space. And strictly speaking there is no 0 intensity light, but that is as academic as having infinite intensity light. In principle yes, but for all practical purposes you could encode any luminance with any bitdepth/bitprecision you want. You just need to know what you are doing and what assumptions any calculation has.
I am derailing things again, sorry! :grinning_face_with_smiling_eyes:

It goes to achromatic in a nice way. Okay. I would not be too sure about the reasons as stated above. But I think that’s the same reason why people came up with filmic in the first place. To do exactly that. Aurelien hinted that the sigmoid(s) you use might be a special case of the polynomials used in filmic. So there’s that!

the devs of dt will get you up to speed in no time!

Cheers!

1 Like

To be honest, I didn’t try the module (not in a place to be building darktable) and I’m sure I don’t understand a lot of it…

… But it sure sounds a lot like filmic. Map vales outside of [0,1] to values inside of [0,1] while managing the color of it all.

What am I missing?

@jorismak

Yes, it’s like filmic but with only one slider for adjusting the compression instead of 7. The very reason I posted it was that it performed much better than I anticipated and I thought it would be fun to hear the experience of others. So you understood it correctly :slight_smile:

@PhotoPhysicsGuy

Always a good thing to do things like an industry-standard does it.

Don’t get me wrong here. I’m all for not following a standard when there are good reasons for it! But I also think that standards and best practices stem from somewhere and should be studied. I might just need to ask more questions and try things in code to get to that point.

About a one-click workflow in ACES by just changing the ODT, this sounds amazing but I’m not surprised if it doesn’t work in practice. If you have been around the ACES forum some, any particular threads that are worth reading about these things?

Abney effect, nice new thing to learn, kind of makes sense. Any idea how this maps to image editing, do we want to compensate for it to keep perceived hues constant, or do we want to boost the effect for simulating a stronger white light? I can see both options as reasonable.

Aurelien hinted that the sigmoid(s) you use might be a special case of the polynomials used in filmic. So there’s that!

It should at least be similar because they are not drastically different in the results from each other. I’m currently trying to reverse engineer the filmic polynomials to have a better 1D baseline/comparison, will post the results when they are ready.

The reason is that it removes saturation before the rgb compression and less saturation means less hue shift.
This is called crosstalk desaturation for HDR tonemapping, it’s possible to adding back the same amount of saturation.

Another way is to remove saturation only above the reference white point, so for example the desaturation for a HDR image in the 0-10 range will start at 1.0.

However i don’t think using the rgb ratio with a filmic (sigmoid) curve is better than pure rgb and really don’t understand why it’s the default in darktable , it just produces images with very very low saturation in the shadows and midtones, and yes it shifts the hues for the worst.

2 Likes

I am not saying that they want one-click solutions for generating ODTs. They never cared about gamut-remapping because they would regrade everything anyway. This new gamut-remapping they introduce will just let them go from previously loosly defined behaviour to now somewhat better defined behaviour.

Maybe this? https://community.acescentral.com/t/wrapping-up-the-architecture-vwg/3433 It gives a whitepaper of the state that they have reached. There are several different threads with discussion before that though with many more picture comparisons.

1 Like

Keep tabs on

which I typically recommend whenever someone mentions ACES, SSF, etc. May not necessarily answer all your questions but I learned lots.

1 Like