New Sigmoid Scene to Display mapping

Software darktable
418

That is my feeling with some photos.

When there is an overexposed sky it is not easy at all to expand that lights to the mid tones.

You have to make many adjustments and layers in order to expand them a bit.

I have made some improvments, but could not get a pleasent result.

And I am not talking about generating dramatic skies, but natural looking skies with more details and clouds.

In other photos filmic produces superb results without touching anything.
I like it very much for that photos.

I had tried sigmoid some months ago for a while, and it seemed to produce easier to obtain results for that photos.

I am not saying there is no way to get them with filmic, but it is nnot easy.
It is not just me there has been many people with the same problem.

But of course developers may take that into account or not, it is their toy and their time and we will be grateful anyway.

There are a lot of things I like in darktable, and I like filmic too, but nothing is perfect or solves every problem or every case.

But the reality is that there is many people that has problems with skies and with clipped highlights.

2 Likes
419

Thanks for flagging your interest in what I have been tinkering with the last year @ariznaf!

First of all, I have not abandoned this display transform method at all and I still hope it will be accepted in some form or another. I will make sure to push a recent rebase soon so you can use/test it with the 3.8 tools if you want!

There are two things you (who read this thread) can do to help me move this work forward quicker:

  1. Contribute with tests and images where the sigmoid module fails, works, or even is better than filmic. Constructive (both positive and picky about if it is good enough) discussion in this thread fuels my motivation to move forward!
  2. Help out with figuring out how this can/should integrate with darktable. UI, module(s), and code-wise. (This is what I in the end need from the dt devs.)

About options. Every single display transform will have its drawbacks as the point is to compress an infinite space to something finite. The compression implies loss of information and every method has to make a choice in what information it wants to discard. The current filmic module’s choices are good for some images but quite frustrating for others… And don’t despair if you struggle with filmic sometimes, it’s just due to the fact that it isn’t always the optimal tool for the task at hand.

10 Likes
420

Thanks for your answer and for your time and efforts in investigating other paths.

And thanks to all developers of DT. Sometimes when you point some flaw or problem you have it may sound like a negative feedback or seem ungrateful.

But it is not, I have an idea of the time and effort it takes and I am grateful for it. DT is great in so many aspects …

Jakob, I am on windows, I can do tests on it.

But compiling DT in windows is not easy, it has many linux dependencies.
Is there a windows executable, somebody that does the compiling of your sources?

I remember I installed it from an executable, but now I cannot find it in your github place or anywhere.

If there is no option to download an executable, I will see if I can compile it for windows.

Then I will make some test with photos I don’t have good results in filmic and try with both.
@jandren should I put them here or better in a separate thread in the play raw space?
I think it may be better in play raw and here just a link to it, so others can contribute there.

As you say, it is obvious that the information in the photo has to be compressed to fit the output device (screen, file or print) as almost alwasy it has less DR.
So you have to loose details.

In the case of DT and the nonbounded space that compression may be more important.
Filmic opts for giving the most importance to the midtones and compresses highlights with the curve in a way that in some photos makes you have no details in sky with clouds, for example, when you are back iluminated scene.

That is why having other ways of doing scene to screen conversion would be helpful.
Even if there is a way to get it in filmic it is not an easy way and has too many sliders to be able to configure it properly.

Mapping [0-inf] to [0-255] is a bit like projecting the sphere over a plane: there is no perfect way of doing it and each way is suitable for a need.
Cilindircal projections are great for latitudes near ecuator, but of no use if you are organizing a polar expedition.

From a user stand point of view (I don’t know the technical details that seem to make your module somehow not compatible with the new modules) it seems not all that difficult.
This modules is in the final steps, so you just select it and deactivate filmic.
If you like it more, you use that module, and if it creates some eflaws like changing color a bit, you try to counteract it using other tools below it (or above it).
If it gets a better result (to the user taste) with less effort in that photo, you use it.

The tests I did let me think sigmoid was promising being it easier to get better results in skies.

Having to use two installations of DT with two databases renders sigmoid to be used just for testing or very specific use in the most problematic photos, as you can’t be back and forth between both.

If you can select among sigmoid and filmic, and you are providing upgrades, I think a solution would be to use your distributions and have both worlds.

421

It may seem picayune, but all these filmic/sigmoid/basecurve transforms are not the display transform. That occurs in the application of the export/display ICC profile. What you’re doing here is just a tone curve to manage dynamic range.

The two curves cumulatively determine the tone transform from linear to what’s viewed…

3 Likes
422

Yes, you are right.
And that is why I don’t understand how it can be so incompatible or problematic with other prior modules.

If it just comprises tones in a way more simple that filmic it can do it better or in a less optimum way (it depends how you define that optimum) but won’t hurt other aspects of the image.

423

There have been several calls to put sigmoidal tone mapping into the filmic module, which is just a different tone mapper, and nobody has done so yet.

2 Likes
424

Ya I wasn’t even trying to make a counter argument just following on your comment for which I really didn’t have any good background or grounding so I was just curious to see what comments would follow… :slight_smile:

425

This IMHO is where the simple “do this before that” advice breaks down. Now that we have the ability to mess with order of operations in darktable a whole 'nother layer of knowledge is needed to understand what orders are better than others, and what orders will just muck things up.

The whole generic raw processing pipeline has two distinct phases: 1) BD - Before Demosaic, and 2) AD - After Demosaic. In BD, the image data isn’t really renderable, but there are things that are usually done here, like black subtract and white balance (for you RT folk, I do WB once, before demosaic). At the act of demosaic, the image data is transformed into an encoding that can now be regarded on RGB media, but the data is still what we call “linear”, or “energy-linear”, or, or, “Scene-Referred”. It is here that processing starts to be discretionary, but there’s yet another rubicon to be considered: Departing Scene-Referred. This is where the first non-linear tone curve is applied, destroying the original energy relationship of the measured image values.

It used to be de-riguer for softwares to immediately apply a lifting tone curve after demosaicing, so the operator could regard their image nicely and feel good about the software. But, then the software would allow one to apply color and tone manipulation on top of that initial tone curve, and that’s where the problems were introduced. So, it’s not that one particular tool should go before another, it’s that one should do any color editing before the image data is yanked from linear, and preferably that linear departure should be as close to the last thing done as possible. This is the basis for the whole migration of darktable to scene-referred editing: putting as much of the editing as possible before the departure-from-linear.

My picayune point was that the tone curve you manipulate in the darktable UI is not the only one applied to make the final rendition. The export/display ICC profiles have one too, and it is applied after and on top of the output of filmic/sigmoid/whatever. So, what you’re messing with in filmic/sigmoid is the place between linear and the start of the rendition transform…

7 Likes
426

I think there are two different concepts for which this would apply,

  1. Make it part of the filmic module itself.
  2. Make a “meta” module that holds all 3 (or more) transforms into display dynamic range into one common module, but keep them apart such that only one is active at a time.

The second would be just a UX thing and technically they would be three different modules (I consider base curve number 3). Having such a meta module makes IMHO sense as you typically have only one display transform, and you decide between filmic, basecurve and this approach. It would unclutter darktables UI a bit, without reducing flexibility. In this meta module, you just select filmic and get access to all filmic settings/UI, or you select basecurve and get basecurve’s UI, etc. Only one of the three would of course be active, the one that is selected. For the rare cases where you need 2 basecurves and 1 filmic module, you could just use the typical module mechanisms.

IMHO, 2 makes sense, 1 does not as it would complicate filmic.

Edit: Pipeline position could be the same for all three, or, if it is really required, default pipeline position could change when another method is selected, unless there was a manual shift of the pipeline position by the user, which would override the default position.

5 Likes
427

Thanks by the great exposition of the general workflow.

I knew it more or less in general terms, but your clear explanation is great for refreshing concepts.

But the question remains the same: why you can’t just substitute filmic tone mapping fro another one like sigmoid (or upcoming ones).

All it should do is map [0-inf] to [0-255] there are many ways of doing that (few make sense probably) but it is a bit like selecting a mapping projection or another, best depends on the use you aree going to make with the plane and the part of the Earth you are interested on.

If one does not suit you, you can use another.
If it is just tone mapping it does not seem problematic to substitute that tone mapping curve.
After that you can do color adaptation for the display.

To make its job in the best possible way, the tone mapper should not change hue or saturation or do it as few as possible.

But reading the answers it seems that filmic does not only makes a tone mapping.
Indeed I had the feeling that it bites too much, as it tries to recover higlights, there are options to preserve colors in “clippped” highlights…
shouldn’t that other operation be separated from tone mapping task?

428

No reason you can’t, except for the pull request hasn’t been accepted… :laughing:

Sometimes when developing a high-DR scene, like with shadows at sunset, I’ll delete my default filmic curve and replace it with 1) a loggamma curve, which lifts the shadows into a milky midrange, and 2) a regular control point curve that I twist and skew to make the image look nice low-to-high. It’s usually S-shaped to pull the shadows back down, and keep the highlights from washing to white. That’s the fundamental problem with a filmic-shaped curve: the shoulder at the top can’t be flattened too much, even if sometimes it should be…

429

Well, yes, it is a small detail xD

But there are some arguments about that, saying that it has some collateral effects and does not integrate well with other new modules.

But it is at the end of the chain and just does tone mapping it cannot have collateral effects in other modules.

Well I will try to compile DT and later integrate sigmoid in it.

It will be the way of being able to test things and modules not in the main stream, as there is no plugin mechanism in DT.

Yes that is exactly my feeling sometimes with filmic sometimes and it renders all your highlights (skies) in few space at the output.

That is why having an alternative of mapping values is great.

1 Like
430

I want to remind people that there are only 2.45 EV of available DR between middle-grey and peak SDR white. Whatever tone mapping method you choose, skies are going to be compressed in display because they have a lot more than 3 EV DR on the scene. Which, by the way, is consistent with human vision.

Let’s get some reference:

  • Middle-grey is the luminance of a 20% black patch at 20% reflectance, and records at 18.45% linear luminance display-referred SDR,
  • color checker reflective white at 20% reflectance records at 92% linear luminance display-referred SDR.

What this means is, at 1:1 contrast ratio over reflective surfaces, you are left with the upper 8% of the SDR display range to squeeze all pseudo-emissive clouds and lightsources. If you increase contrast in reflective areas, then reflective white will register above 92% and will eat the emissive range accordingly.

So much for dramatic skies. We simply have no room for them in SDR.

The other option is to pull down the middle grey level, which means darkening the picture globally, and that’s not what you want to do.

Whishful thinking changes nothing to this fact, until HDR displays become mainstream. Different tonemapping strategies will yield different curvatures and slopes, but if you anchor black, grey and white and derivate a smooth transform between them, you will get mostly the same results.

Getting more contrast in HL needs to be achieved by local tonemapping, at the risk of breaking luminance relationships between sky and ground (light rources are expected to be brighter than reflective surfaces) and introducing usual HDR creepiness.

All of that wouldn’t be a problem if people didn’t got used to overdone tasteless skies born of happy slider pushing. Remember aesthetics are acquired.

Photography and videography are technically bound arts, and if display can’t display what you want it to display, no matter how famous and praised you are, you need to reconsider your creative choices.

6 Likes
431

Of course you are right, and may be skies in photos are not as natural as we remember.

Or may be we do not remember the sky the same way our eyes see and capture it, as vision is not a subject of phisical light and “eye” response, but a matter or interpretation in our mind.

When we see a sunset we don’t capture all scene at a glance, we concentrate in different parts, move the eyes all over the scene while they adapt to differente levels of light and keep in in our memory after that.

But it is not as important to discuss if it is more or less real what filmic or other mapping tools get or whether skies in many photos are not so real.

Photography is not just capturing the real scene, but an artistic interpretation of it.

It is usual to lower light in the sky to enhance details, make selections and change color a bit.

Is it real? not for sure, but reality is boring, and may be your sky was not the best at the moment of the photo, and you want to “enhance” it a bit.

The problem is not that filmic comprises highlights, it is expected to do it, any simple contrast curve does it, and that is the use of filmic.
The problem is that it is not easy to revert it, even with curves and parametric slections, or using tone equalizer (it seems to work the best) or color balance RGB to lower highlights.

I don’t know exactly why it gets so difficult.

In other programs (due to non linearity of their workflow?) it is easier.

2 Likes
432

That’s because it is a local tonemapping issue and you try to tackle it with global tonemapping methods. Wrong approach, wrong tools.

Try dodging and burning, it has worked since circa 1860 to solve this. Tone EQ is one way of doing it, masked exposure is another. Aka decrease exposure selectively on a region.

3 Likes
433

Yes that is what I am trying now, using exposure to reduce it a bit (with masks) and tone eq.

It improves, no doubt of it.

434

You might want to also check out the recent work in RT…I have not had time but it sound interesting

Log Encoding

The code used in this part of RawTherapee is similar to:

  • The Log Tone Mapping module in ART, designed by Alberto Griggio.
  • The Filmic module in darktable, designed by Aurélien Pierre.

Both are inspired by the work on logarithmic coding developed by the Academy Color Encoding System (ACES).

The algorithm is based on a 3-step process:

  • The first step for a given image (HDR or otherwise) involves calculating the deviation from the theoretical mean gray value (18% gray) of the darkest blacks and the brightest whites. This is expressed in photographic Ev units (luminosity index, which is related to the brightness of the scene). The black and white Ev values, along with the average or mean luminance of the scene (Yb%) are used by the algorithm (either automatically or with manual override) to modify the balance of the RGB values, thereby reducing contrasts, enhancing shadows and reducing highlights, without overly distorting the image rendering.

  • In the second and third steps, the data is manually corrected by the user to increase local contrast (which has been reduced by the “Log” conversion) and adjust the viewing conditions for the intended output device.

https://rawpedia.rawtherapee.com/CIECAM02#The_algorithm_is_based_on_a_3-step_process:

2 Likes
435

As primarliy a RT user I’ve used the Local adjusments > Log encoding quite a lot building mainly from dev.

Despite the explosion of sliders in the RT interface the decoupling of the log and the curve makes it easier to control imho. It does suffer, when used as a full image edit, from destroying highlights in clouds! I frequently have to add an excluding spot over highlights to bring back detail. A very different issue than the darktable one but perhaps they are related? In RT it seems to sort of undo any highlight recovery and flattens it do a dull colour. It’s my pet peeve with RT Log at the moment.

For those interested I basically use

  1. Log strength
  2. Mean luminance (under viewing conditions. I (ab) use this to raise shadows)
  3. Contrast

Above are in the order I tweak them.

436

If someone has the time, it would be great to have a “this is how you get dramatic skies in Darktable 3.8” tutorial.

Basically a RAW file shot during daytime with some blue skies + nice clouds, a mild exposure adjustment with a gradient mask, then diffuse and sharpen and finally some color boosting to taste.

It really got much easier in 3.8.

2 Likes
437

Well I had a look to RT and ART.

They are good, but not the kind of program I was looking for.
No tools to manage photo collection, slow screen refreshing when you change an slider and no feedback when you are moving it, clutter interface…

But for my tests they make a better work in clipped light recovery at least when recovering color of zones witch one or two clipped channels.
And it is easier to get more pleasent results in highlights, may be due to the more classical non linear approach to developing.

In general I far prefer DT , for it it only has these drawbacks: poor clipped light recovery when you need to reconstruct color and harder to get pleasent results in highlight when you want to expand them to the midtones.