New Sigmoid Scene to Display mapping

[Tamas_Papp (Tamas K. Papp)]

@ariznaf

Well yes, but that was an easy one, not many contrast in the original.

[…]

When you have a backlit and your sky is a bit overexposed in order to have no too strong darks, and you need to compensate both gets harder.

So (again, and sorry to harp on this): an image which you (or someone else) considers challenging in Darktable 3.8 for the purposes of this discussion would really help to focus the conversation.

Without concrete examples, is it very hard to say if something is inherently difficult or unsatisfactory in Darktable, or perfectly possible and easy with the right tools.

Personally, I think that it will take me another 6–12 months to transition to the new setup in a way that I can exploit its full power, so I am marking photos to revisit with the amazing new stuff I keep learning here.

[ariznaf (Fernando)]

Yes, you are right.
I am right now seeking and trying with some of mines.

I had previously linked an image that I considered it to be quite challenging, as it has some clipped lights in a yellowhish zone.

And it is backlitted: a bridge over the sky.
It was uploaded to a spanish forum in order to get edition advices and nobody could get a very pleasent result.

But it is not mine, and so, I cannot put it here (I have asked the author if it would mind to put it here, but received no answer yet so I am seeking for some good sample of my own).
Quitar el cielo morado - Darktable en general - darktable en español

I will try to edit it first using filmic, then sigmoid and compare it with ART (I am not proficient at all with it) or may be my old LR if I remember how to use it yet.

May be it would be better to put it in play raw and just post a link here.

Other situation that takes me a bit of time to fight is when you take a photo of a building against a grey and cloudy sky and you over expose it a bit to get more details in the building.

It takes quite an effort to get details in the clouds.

[priort (Todd Prior)]

I think for sure you work on some skies at least in one if not more than one of your videos as well…maybe the one where you explore scene vs base curve and the “jpg” look…also I think your older one on the channel mixer maybe…

Time to go back and do some review for me…

[ariznaf (Fernando)]

Well here is the test I was asked for:

Processing skies with clipped channels in Darktable filmic/sigmoid

Well, this test raw is originated by this discussion about sigmoid and filmic tonal mapping. There is people like me that says filmic is difficult to fight when you have skies that are over exposed and you want to get them back to a more natural and interesting look, and other tone mapping tool like sigmoid would be interesting. So here am I trying to provide an example with overexposed sky (and some other part of the photo) that wanted to turn in a more natural look. Not a good or interestin…

Probably not the best testing photo you can get (the other of the bridge would be more interesting) but one that presents clipped highlights and a dull overexposed sky.

I have processed it with sigmoid, but I have an old version (DT 3.3).

The results using color calibration to darken sky I think are quite better than what I was getting previously.

Thanks to all.

[paperdigits (Mica)]

@ariznaf

When you have a backlit and your sky is a bit overexposed in order to have no too strong darks, and you need to compensate both gets harder.

You need to stop clipping your highlights. Shadow recovery is amazing with the scene referred workflow.

[ariznaf (Fernando)]

When you have a backlit scene of a building you want good detail and exposure in the building, sky is not the most important part, but you would like to have a sky with some levels not just an almost whit sky.

In sunsets, not clipping some part of the image is not an option (even if it is the sun, which is not white by the way and which would be problematic in DT).

Other situations of low light with intense ilumination like pubs or discotheques or concerts are other examples of when clipping lights is mandatory.

And not everybody has the ultimate sensor in his camera in order to be able to recover shadows several steps with low noise.

Mine are not bad, but xe2 is not the best in that aspect.

But I will keep it in mind, may be that in scene referred workflow you can recover shadows better and with less noise compare to the old tools we were used to.

[paperdigits (Mica)]

Then you should bracket when shooting a building or sunset.

Really hard to have dramatic skies when you’ve clipped the highlights.

[ariznaf (Fernando)]

Not me who was asking for dramatic sky (depends i what you called dramatic) just not so dull.

You don’t allways have a tripod or the time to do that.
Most of my photos are when travelling or going for a walk, not a prepared photo.

That would be an appropiate answer if other tools were not able to get better results.

What was discussed here was whether filmic and /or scene refered workflow was in the origin of making it more difficult to expand those lights or not.

The color balance trick provided by @s7habo seems to work quite well.

And highlight recovery in color preservation is problematci in DT as it creates artifacts.

Just addressing people to not clip their photos would not solve the flaws.

I think we are going a bit out of topic here, and @jandren won’t be happy.
The topic was about comparing sigmoid and filmic and whether it would be good to have it in mainstream.

Most of us are having a look to sigmoid due to difficulties in expanding highlights (in skies for example) and seeking for alternate paths.
May be related but not the initial intend of the OP.

[rvietor]

But when an important zone in the image has clipped pixels, you have irretrievably lost information. Any program that claims it can recover that information is lying, plain and simple. The best you can hope for is some kind of approximation. And getting a good colour approximation is even harder than getting an approximation of luminosity, even when only one channel is clipped (with three channels clipped, I wouldn’t even talk about “approximation”, it’s more like “guess”).

And I don’t see where switching between sigmoid and filmic has any relation to highlight recovery…

[ariznaf (Fernando)]

Of course you have lost information, but some tools do a better job recovering or giving the photo a more natural aspect than others.

Most times the clipped areas are more or less uniform in color, and the luminance can be retrieved from the remaining channels.

The color preservation in highlight recovery does a decent job… if you overlook the artifacts it generates many times.

You are right, there are too separate things: highlight recovery (nothing to do with sigmoid) and the difficulty in expanding highlights to the midtones when using filmic and the linear scene referred workflow compared to the traditional workflow (where sigmoid or other tone mapping tool could help, or at least some would expect so).

The debate about recovery emerged when it came to give examples about skies.

[nosle]

@paperdigits

You need to stop clipping your highlights. Shadow recovery is amazing with the scene referred workflow.

This is only true to some extent. Scene referred doesn’t improve your camera sensor. Under exposing means noise. Filmic can handle raising shadows quite gracefully but it introduces just as much noise as other software because it’s in the data.

Often the sky or other blown areas are of tertiary importance. There’s no real info there for your scene but you just don’t want it looking odd and attracting undue attention. Graceful handling of highlights mean you can trade in noise for recovery in areas of quite low and unimportant detail. Most software makes this a possible trade off.

Edit: Just to add that I write this as a recovering clipophobic. I have thousands and thousands of underexposed images.

[paperdigits (Mica)]

@nosle

Graceful handling of highlights mean you can trade in noise for recovery in areas of quite low and unimportant detail. Most software makes this a possible trade off.

You can’t gracefully handle what isn’t there. That is my point.

[s7habo (Boris Hajdukovic)]

@ariznaf

the difficulty in expanding highlights to the midtones when using filmic and the linear scene referred workflow compared to the traditional workflow (where sigmoid or other tone mapping tool could help, or at least some would expect so).

Just consider the recommendation @anon41087856 gave regarding the difference between global and local tone mapping.

With the global tone mapper (filmic) you treat the whole dynamic range and with local (tone equalizer) you treat certain brightness ranges. This allows you to treat brightness ranges separately.

Example:

This photo is underexposed because the highlights on the walls of the houses should not be overexposed due to the very large dynamic range:

If I increase the exposure to see the market place better, the highlights are overexposed:

With filmic I have compressed the dynamic range and accordingly the highlights are there again, but the contrast there has been lost through the compression:

This is also logical because almost 4 EV in highlights have been “squeezed” from the dynamic range of the camera into 1 EV of the dynamic range of the screen:

To get contrast back there, I now have to treat this area locally. So I have to,

1. move it into the middle grey (darken it)
2. “spread it out” from there to the highlights.

So, after the white balance, I can first darken the sky with the help of the second instance of the colour calibration module and thereby increase the contrast between the sky and the clouds:

I can also use colour balance to increase the contrast even more:

And now I can use Tone Equalizer to treat only the highlights (walls of the houses) separately to increase the contrast there. First darken them (move them into middle grey) with first instance:

And then, with the help of the second instance, without perservation of the details (which means, to use it like a normal tone curve), I can expand this area back into the highlights:

And last, I can use diffuse and sharpen module to increase local contrast a little bit:

[nosle]

Well most raw software can so it’s clearly and empirically possible.

[paperdigits (Mica)]

@nosle

Well most raw software can so it’s clearly and empirically possible

Nope. When channels are clipped, you loose that data.

Sure you can guess what should be there and put data in there, and darktable is weak here (but new goodies coming), but its still a guess and the application is inventing data that isn’t in the raw file.

[anon41087856]

@nosle

Well most raw software can so it’s clearly and empirically possible.

Local tone mapping, and their followers : the halos.

[nosle]

I think this discussion has slipped into highlight recovery which is part of the same problem of managing highlights but is still a distinct issue that I know you are working on.

Filmic is getting increasingly better but it’s still difficult to control the highlights of the global tonemapping. I agree that you have limited “space” to do so but It’s usually enough and workable with a more flexible curve. I understand that filmics answer to the issue is local tonemapping with the tone equaliser?

[paperdigits (Mica)]

@nosle

I understand that filmics answer to the issue is local tonemapping with the tone equaliser

Yes, or masked exposure module.

[anon41087856]

@nosle

I agree that you have limited “space” to do so but It’s usually enough and workable with a more flexible curve.

OGM NOOOOOO. I have explained that, and Boris has shown you the graph.

Stop with wishful thinking, any curve will give you the same result because it’s constrained in 3 points and display peak white is 100% while diffuse reflective white is 92%. 8% is 0.12 EV for specular highlights and emissive stuff.

No bloody global mapping, no curve, no one-size-fits-all method will give you that.

USE LOCAL TONE MAPPING.

[Underexposed (Bill Martz)]

@s7habo

second instance of the colour balance module

Second color calibration?