Processing skies with clipped channels in Darktable filmic/sigmoid

[ariznaf (Fernando)]

Well, this test raw is originated by this discussion about sigmoid and filmic tonal mapping.

There is people like me that says filmic is difficult to fight when you have skies that are over exposed and you want to get them back to a more natural and interesting look, and other tone mapping tool like sigmoid would be interesting.

So here am I trying to provide an example with overexposed sky (and some other part of the photo) that wanted to turn in a more natural look.

Not a good or interesting phot, just for trying, a photo with that kind of problem.

DSCF0175.RAF (24.6 MB)
This file is licensed Creative Commons, By-Attribution, NonCommercial, Share-Alike.

This is what I was getting, trying to revert the overexposed sky and get a bit more interesting sky with a natural look.
I used to employ color balance RGB and masks.

DSCF0175.RAF.xmp (12.1 KB)

No good details in sky.

We were adviced to use color calibration with masks to brighten red and darken red in sky, and tonal equalizer to adjust tones (not needed in this photo, I think).

I have tried and I have to admit it seems to provide very good results, at least in this photo.

DSCF0175_01.RAF.xmp (17.6 KB)

A problem is with the highlight recovery.
This photo has areas of clipped blue and green but not too difficult to reconstruct with other softwares, as it is quite uniform in color in that zones.

I have activated highlight recovery in color recovering mode, in order to recover the color and don’t get it grey.

It seems to do what intended, but creates some artifacts.
It seems a recognized problem and I was said it is being worked out with a new module based on guided 4th order laplacians so not problem, as it will be corrected.

Finally I have tried it with sigmoid.
I have only an old version based in 3.3 of DT and not 3.6, so I could not make the same exact processing.
This is what I get.

It would be great to see others results and comparison with sigmoid and filmic.
And results with RT too, in order to get a reference and see what can be obtained and with how much effort.

DSCF0175_02.RAF.xmp (23.6 KB)

May be there are much more better ways of developing it using filmic or sigmoid.

In particular, filmic seems to provide a way of recovering highlights on itself.

To do that, if I understood it well, you just do not use recovery at all in the base module, so filmic will receive overexposed values over 1.0 and compress them to 0-1 and try to recover them.

But I have tried with several photos to do that and get no good results, as it seems to turn them to a grey/white value with no propagation or surrounding colors.
I have to admit I don’t understand well that reconstruction tab in filmic.

May be @anon41087856 or others can show us a better way of using filmic in cases like this.

[paperdigits (Mica)]

I think you’ve misunderstood filmic’s highlight reconstruction and I’d urge you to read: darktable 3.8 user manual - filmic rgb particularly the part about how it does not reconstruct or inpaint large clipped areas. You need to try the Highlight Reconstruction module for that.

[kofa (István Kovács)]

filmic with auto-set white relative exposure, latitude pushed to 90%. highlight reconstruction turned off.

DSCF0175.RAF.xmp (7.8 KB)

Could be pushed further by adding local contrast in multiply mode, masked for highlights (trick from @s7habo ):

[priort (Todd Prior)]

Also it would seem at least here luminance color preservation mode of filmic retains the most blue in the sky…

EDIT

Default on open…

Just changing the norm to luminance…

[Tamas_Papp (Tamas K. Papp)]

Note that since you didn’t post your image under a permissive license (eg CC-BY-SA), and in fact have Todos los derechos reservados under Rights in the metadata, you are not really encouraging edits.

[ariznaf (Fernando)]

Well yes, that was one of my past tries, using multply, curves, tod expand that lights.

But I have found color balance to be good in darkening the sky and adding a bit more blue to it using the blue and red channel and a mask and parametric más for the selection.

It is an easy way and it produced good results, I think (second image).

@priort

Also it would seem at least here luminance color preservation mode retains the most blue in the sky…

I have used colro preservation in highlight recovery, it works but create artifacts.

[ariznaf (Fernando)]

Sorry, it is as it by default in my photos.

May you provide me a littlel advice on how to put the licence here?

I have asked for editions explicitly, so it might be enough.
But providing a licence in the post might be better.

EDIT:
Ok I have copied from another thread.

It is differente giving a license to share your photo for non commercial use than letting people provide examples in a thread in some forum.

In play raw is it not crear enough that if you share a photo here is searching for alternate editions, without the need to force you provide a license to share your photo?

May be a pinned post stating that if you upload an image here you authorized other to use it to provide alternatives just in you post would be a general solution.

It is not important for me in this photo, nobody would be using it, but may be the difference is important in other cases (if there appears somebody of your family or other situations).

[Tamas_Papp (Tamas K. Papp)]

Just edit the original post to say that the license is eg Creative Commons 4.0 BY SA.

[priort (Todd Prior)]

@ariznaf

I have used colro preservation in highlight recovery, it works but create artifacts.

This was in Filmic I was referring to not HR…sorry if I was not clear…so not power norm but luminance

[nosle]

A quick RT processed image for reference. Only a couple of clicks, the look and vertical verticals are just how I happened to do it. No intent really just a rt processed file to compare to. I did desaturate highlights to avoid blue clouds. Not used to the camera and it’s files so the look is a bit odd. Done “at work” so on windows with an old dev build 5.8-3046-g5268b94cf

DSCF0175.jpg.out.pp3 (16.7 KB)

[dqpcoxeas]

My version…

Darktable 3.9.0~git24.477bc5c41a-1

DSCF0175.RAF.xmp (20.9 KB)

[s7habo (Boris Hajdukovic)]

DSCF0175_03.RAF.xmp (13,9 KB)

darktable 3.9.0~git143.8e94cae089-1

90% of the time went to the reconstruction of data.

My warmest recommendation: pay attention to the exposure when taking photos.

[david]

Apologies - I know this is not what you asked, but I could not resist seeing what I could do with this photograph using GIMP.

[ariznaf (Fernando)]

OK, thanks.

But then what sould I use in highlight recovery module?

I have tried in filmic to change to Luminance Y with color preservation in the other module it renders the sky a bit too blueish (and there are the artifacts)

Disabling it or using other options create magenta casts in the cloud.

And what about the threshold and transition and other sliders in lights reconstruction in filmic?
Have you adjusted them?

[kofa (István Kovács)]

You can often completely turn off highlight recovery.
I think it was more important to have it enabled with the display-referred workflow, where having pixel/channel values > 1 is impossible.
I either leave it off, use it in reconstruct in LCh mode, or, very rarely, in reconstruct color mode.

With the module disabled, magenta highlights can be handled in filmic, for example by desaturation (due to being outside of latitude) or on the reconstruct tab. You may need to raise iterations of high-quality reconstruction on the options tab.

[priort (Todd Prior)]

I really didn’t look that close for artifact and you could pull back the blue if it was too much by either sliding the highlights/shadows slider adjusting the latitude or dropping the midtone saturation a bit…or some tweak that combines one or more…this is not really a local edit which is what you want to tweak your sky but rather a global edit that seemed to have less saturation out of the gate…editing the sky is tricky as personal taste is a huge part if you want a rich deep blue you lose a lot of the wispy trails in the clouds if you go too light or too much contrast then you can lose color an blow those out…

I guess it all depends in the end…again from some very minimal messing around the whole sky is in a similar tonal range so when you try to darken it the clouds be come an unnatural gray or they have color if you try to saturate the highlights or add too much color with multiply…

Many of these things can look unnatural until the opacity is backed off to ease into the effect.

Personally its a very bright scene from what I can see and the clouds seem quite diffuse so really I would just massage that a little and call it a day as a good look for me…others might need a different version

You could try this for fun …so set the midtone saturation to full…it will be very blue…now experiment by changing the latitude and moving around the highlight/shadows slider…the smaller the latitude the less saturation and the more you can slide it up and down the curve…

With some combination of these I got a nice deep blue in the sky the clouds were a little puffy and the rest of the image looked not bad as well…I didn’t save it I was just pushing the limits of everything as I often do when I experiment…this is likely breaking some rule but it was an interesting effect…

[g-man]

ISO 200 1/90 at f9.0 in a bright sunny day. I understand trying to use software to recover some clipped highlights, but this is too much overexposure in my opinion.

[Sunhillow (Hansgeorg)]

@g-man

ISO 200 1/90 at f9.0 in a bright sunny day.

IIRC ISO 200 is the base sensitivity of the X-E3. But you are right, shutter should have been faster

[ariznaf (Fernando)]

I have stated that I am not showing the photo as a good photo or a way to show how to do exposure.

It is shown here in play raw and selected as a photo with overexposed sky with areas of partially blown channels.

ISO 200 is the base ISO of this camera.
Its sensor is not the best in DR and noise in shadows.
So with a subjet in the shadows you cannot expose to the sky.
It could have been exposed a bit less, of course, indeed it should be done a point or so.

I am not sure of channels being clipped as much as it seems, may be an effect of color balance, I would like to see the real raw histogram.

The sky in this photo is not so difficult with old workflow in programs that make a good color preserving recovery.
The cloud is quite uniform and only need as bit of blue and luminosity transitions to keep it natural look.

So let us talk about processing photos with this kind of problems and how they can be best treated in DT.

@s7habo

darktable 3.9.0~git143.8e94cae089-1

90% of the time went to the reconstruction of data.

Good result, I have to study that with care.

And I will provide other photo with a non blown sky but in a cloudy and rainy day that would like to dramatize a bit (the sky) and it was not easy to do.

[age]

@ariznaf

There is people like me that says filmic is difficult to fight when you have skies that are over exposed and you want to get them back to a more natural and interesting look, and other tone mapping tool like sigmoid would be interesting.

Sigmoid it’s faster but for not very high dynamic range is possible to get the same results with filmic.

DSCF0175.RAF.xmp (15.2 KB)

I like natural/neutral looking photo so this is my edit, sky is highly compressed:
white balance, vibrance, remove saturation for very saturated colors, rgb per-channel filmic .