Learning to recover blown out skies

[dru]

I am learning to use Darktable to edit raws from a Sony a6000 and to learn more about photo editing in general. Most of the editing I am doing so far is recovering detail when I shoot scenes with high dynamic range (I think?) where the lighter areas get blown out. I have an image that seems to have a lot of data for the sky that I can see with the highlight reconstruction mask in the filmic rgb module:

but my best attempt has gotten me here:

I am still struggling to keep my skies away from Magenta and I can never seem to recover the detail present in the mask. Someone on #darktable on OFTC suggested that I upload the RAW here to see how someone more seasoned might do it! How would you repair the sky?

DSC04603.ARW (24.2 MB)
DSC04603.ARW.xmp (18.1 KB)

This file is licensed Creative Commons, By-Attribution, Share-Alike

[Terry (Terry Pinfold)]

I am downloading your image and will find time to develop it. However, I have personally abandoned the use of filmic because it is difficult to use and to teach my students how to use it and can definitely produce magenta skies in some images. I have moved to using the sigmoid module which gives good results straight out of the box and often doesn’t require any user input for most images. Some images like sunsets definitely benefit by experimenting with the hue preservation slider in sigmoid and the skew slider can influence the contrast and compression in the shadows and highlights.

[123sg (Steven)]

This is the best I could do.
Ideally, if your ISO is low, you’d underexpose a bit on this kind of shot - it’s an awful lot easier to recover shadows.
The problem is that two out of three channels are clipped in that brightest area.

My approach was to first lower the exposure way down to make it easy to see whats happening in the sky, then fiddled with the highlight reconstruction module.
Switching to segmentation helped a bit.

Then I process the image normally, but there was arguably still a slight pinkish hue in the sky so I used an instance of color balance rgb with a parametric mask to add a little blue there.

Personally, I’ve never had much luck with the filmic reconstruction, FWIW.
DSC04603.ARW.xmp (10.2 KB)

[Terry (Terry Pinfold)]

I have just had a quick look and Filmic V7 is producing a magenta sky. However, Filmic V5 with no chrominance preservation is not and Sigmoid is not. I will look at this image in more detail and edit it later. I still recommend considering using Sigmoid while you are beginning your journey of learning DT. It was developed or added to Dt after filmic and I see it as a big improvement in making DT more appealing to the masses. Filmic is very complex which for some people might be good but for newbies is a big challenge.

[123sg (Steven)]

@Terry

However, Filmic V5 with no chrominance preservation is not and Sigmoid is not.

That’s mostly because V5 filmic and sigmoid desaturate the top end, whereas v6 and v7 don’t, instead doing their utmost to preserve whatever colour is there. Edit: as pointed out by @priort below, V7 does allow desaturation.

I used sigmoid, however after pulling the highlights down (as it seems the OP wanted detail there) a bit of magenta returned, as it wasn’t being fully desaturated.
My 2c FWIW.

[commutergraphics]

DSC04603.ARW.xmp (14.7 KB)

nothing super fancy like masking mostly tone equalizer and I sometimes use both local contrast and diffuse and sharpen with the local contrast preset on the same image

desaturated highlights a bit but you could use color balance rgb or zones to remove magenta in highlights

[Sean_Liu]

DSC04603.ARW.xmp (9.1 KB)

As @Terry mentioned, use filmic v5 without perserving chrominance would not make a magenta sky.

[Tim (Tim Cuthbertson)]

The main thing I tried was two instances of Tone Equalizer. I don’t know whether this is a success or a failure.

DSC04603.ARW.xmp (8.6 KB)

[priort (Todd Prior)]

THis is where if v7 is used the highlights slider comes into play…usually you can use it and the magenta would go away but I still prefer and use v5.

Relative and perceptual edits…no filmic or sigmoid to keep as much detail as possible…

DSC04603.ARW.xmp (19.2 KB)

I would likely go back and desaturate the sky

[123sg (Steven)]

Thanks Todd - I forgot about that!

[Terry (Terry Pinfold)]

Regardless of module used lowering the clipping threshold in highlight reconstruction module usually controls magenta highlights problem. The settings shown here may not suit your version of DT as I am using V4.7.

I would add that in V4.7 I am finding some images that previously had horrendous magenta highlights and clouds in early DT versions now look fine and even those that still have problems it is much less than previous versions. I presume just continuous improvements from the developers. I almost exclusively use in inpaint opposed option in highlights reconstruction.

[Terry (Terry Pinfold)]

This is my attempt using sigmoid module. My xmp file may not be too helpful as I am using V4.7.

However FYI I did the following:

I darkened the image to optimise the sky and to hell with the shadows going dark.

I created a second instance of exposure and used a parametric mask to select the sky and then inverted the mask. This allowed me to separately adjust the brightness of the foreground.

I did a small threshold tweak in the color reconstruction module to get rid of some minor magenta cast.

I activated the shadow and highlights module to default values because I like the effect.

I applied local contrast to the default values because again I like the effect.

I activated the lens correction module which corrected distortion and the chromatic aberration.

I applied denoise (profiled) at default values

I activated the diffuse or sharpen module using the preset 'sharpen demosaicing: AA filter

I did a small tweak to the skew slider in sigmoid (+0.26) to add contrast to the sky without blowing out the clouds.

Then in the color balance rgb module I added about 16% vibrance, 7% contrast, +38% brilliance in the shadows and to further tackle any unwanted color in the sky the decreased highlights saturation by 51%.

This took less than five minutes to do even though it might sound complicated. I am pleased with the result, although the posted image is a little darker than on my screen in DT. This sometimes happens when I post images to the forum.

BTW, @Sean_Liu has done an nice edit using filmic V5 with no chrominance. There are lots of people who like filmic and when I use it I tend to use V5 with no chrominance. But it is a complex module to learn and I recommend Sigmoid because it is very intuitive and easy to use.

DSC04603.ARW.xmp (16.9 KB)

[Terry (Terry Pinfold)]

@123sg

My 2c FWIW.

You will have to change that to your 5 cents worth as two cents coins are no longer legal tender in Australia.

[123sg (Steven)]

Oh… I must be out of date!

[kofa (István Kovács)]

The best way to deal with highlights is to preserve them when shooting – that is, avoid blowing them out, by reducing in-camera exposure. Exposure then can be restored in darktable – in fact, the exposure module has an option to do that automatically (I took this screenshot using a photo I took in manual mode, so there was no exposure compensation dialled in the camera, but you get the idea):

The reconstruct option in filmic is somewhat of a misnomer, as the primary purpose was to soften the transition between what you set as white (white relative exposure on the scene tab) and the surroundings. It is not there to fix raw clipping: filmic has no idea what raw pixels were clipped, it’s close to the end of the pipeline.

In your photo, turning off all curves (incl. filmic) and dropping exposure shows the magenta is there; it’s not added by filmic, but it may be emphasised by it, depending on the settings:

Turning off highlight reconstruction (the module, not the option in filmic) shows what we start from:

Turning on the raw clipping indicator also confirms the severe clipping; you have lost a lot of information, and can only try to cover up that fact. Restoring lost information is impossible.

Turning on the candidating mask shows which areas get some colour propagated from the surrounding area:

With the default value of 40, only a bit of the blue channel is propagated:

Read about the meaning of the masks and settings, as explained one of the developers here: Nice waterfall, shame about the vegetation blown highlights - #22 by hannoschwalm

But there is not much you can do:

Side-by-side with the default inpaint opposed:

exposure raised back, filmic back on (v7 with the defaults, only auto-picked the black and white exposures):

With my default filmic style (includes diffuse or sharpen and local contrast, too): filmic v7, but with contrast in highlights/shadows set to safe for a gentler curve:

Left: default filmic v7, right: as described above:

Bringing down the highlights a bit in tone equalizer, then dropping highlights saturation mix in filmic:

DSC04603.ARW.xmp (15.2 KB)

The sidecar is from darktable master branch (development version).

[qmpel (blends colors 🎨)]

Here is my version with ART, using balanced highlight reconstruction, dynamic range compression and tone eq with a bit of local contrast…

DSC04603.ARW.arp (13.5 KB)

[kofa (István Kovács)]

Wow! I’ll try to see if I can come closer to that when I’m home.

[arturoisilvia (Arturo)]

With DT 4.6.1
Filmic V6 No + Enable highlight reconstruction

20249327_DSC04603.ARW.xmp (8,0 KB)

Greetings!

[Popanz]

Using Sigmoid:

DSC04603.ARW.xmp (14.5 KB)

[Donatzsky (Nis Donatzsky)]

@123sg

Personally, I’ve never had much luck with the filmic reconstruction, FWIW.

As I understand it, filmic isn’t doing actual reconstruction, but is more about smoothing the transition in a way that makes sense for the image. Especially the older reconstruction methods (before inpaint opposed) can have some unpleasant transitions.