Blown highlights in old Nikon NEF (comparing HLR methods)

[kofa (István Kovács)]

darktable 4.1.0+828~g156903b8d

Here is an image with blown areas:

Only basic processing is performed: no tone mapping (base curve, filmic or sigmoid), and no highlight reconstruction. White balance is set using the legacy method (using the white balance modules, color calibration is disabled).

The purple areas show that parts of the statue and the sky are clipped.

Let’s see how each of the recovery methods deal with this.

clip highlights removes all detail, as expected:

The surface of the statue is very close to grey, and lacks features:

The clipped area of the sky is also grey:

reconstruct in LCh restores the texture; the statue is neutral is colour, so this looks quite natural. Left: clip highlights; right: reconstruct in LCh.

The clipped part of the sky is close to grey:

inpaint opposed leaves some magenta:

It fixes the sky:

… introduces noise elsewhere in the sky, because some pixels are below the recovery threshold, and stay the original colour (upper left), while others are recovered:

Reducing the threshold to 0.65 (!) treats the whole sky as overexposed, and fills it with blue:

guided laplacians leaves some blotches:

… and does not fix the sky:

Also, it produces weird artefacts (the sky next to the tree has clipped pixels):

Lowering the threshold and increasing the diameter fixes the magenta, but the green (from the leaves?) starts leaking in:

inpaint opposed, with the same lowered threshold, still leaves some magenta:

segmentation based using the defaults works much like inpaint opposed:

… but does not produce noise at the clipped-unclipped boundary in the sky (left: inpaint opposed, right: segmentation based):

2017-10-15_13-50-59-DSC_0026_05.NEF.xmp (7.1 KB)
2017-10-15_13-50-59-DSC_0026.NEF (20.6 MB)

[hannoschwalm (Jens-Hanno Schwalm)]

The noise with opposed inpaint is more related to non-perfect chroma correction. That’s one of the segmentation advantages.

[kofa (István Kovács)]

For comparison, RawTherapee:

Luminance recovery:

CIELab Blending:

Colour Propagation:

Blend:

[kofa (István Kovács)]

Any idea, why some magenta remains? Is perhaps the raw white point wrong? The old clipping and LCh methods are not affected, though.

[Iain]

Choose your artefacts carefully

[Leniwiec (Michał)]

Nice and informative comparison, thank you!

[garagecoder]

@kofa

Any idea, why some magenta remains?

That’s almost always because the missing area estimation is simply wrong.

This image is a bad case for inpaint opposed because it contains two different coloured highlight clips separated by dark regions - no surprise it fails (it’s a global recovery method).

Perhaps segmentation based could be looked into though… I did some test of the inpaint opposed algorithm on each area individually and the recovery was fine.

[Soupy (Tim)]

Very tricky! I was able to get pretty good results combining methods.

Start with filmic reconstruction.
Combine with guided laplacians with small diameter to help tree edges a little.
Combine with color balance (desaturate, hue mask magenta) was able to remove magenta fringing.
Combine with color balance (blue hue shift, hue mask magenta magenta sky) made the sky look normal.

Some screenshots:

blown-highlights-old-nikon-2017-10-15_13-50-59-DSC_0026.NEF.xmp (18.7 KB)
dt 4.0.1

[g-man]

@kofa

Is perhaps the raw white point wrong?

Compare exiftool to darktable and rawtherapee to darktable.

Since you are going low in the threshold, i suspect it could be wrong.

[gaaned92]

ART

[kofa (István Kovács)]

I wanted to do that, but was unable to find the right params.

[priort (Todd Prior)]

Unless I missed it I don’t see one given…

kofa.txt (11.4 KB)

[g-man]

RawTherapee is using this:

“ranges”: {
// measured at ISO 100. ISO differences not measured, but known to exist
“white”: [ 16300, 15700, 16300 ], // typical R 16383, G 15778, B 16383

I would pick 15700 as the white point, but do keep in mind that this data was only measured for 100 ISO.

darktable via rawspeed is using:

  <Sensor black="0" white="15892"/>

[hannoschwalm (Jens-Hanno Schwalm)]

Just another comment from here. The opposed algorithm is kept very simple to stay performant (it was basically just one part of segmentation) allowing it to be used as a default algo.

Whenever you observe such chroma noise, blown-border issues or alike there is a very good chance segmentation will do better. It needs some user help though. In most cases you just tune the candidates slider. When pressing the mask button beside you can see what parts of the image got a good candidate instead of the global correction in opposed.

[agriggio (Alberto)]

Hi,
I just wanted to say that I’ve been testing the opposed algorithm on some of my images, and I am really impressed by it. It’s so neat and simple that I’m stealing the code for ART too. Kudos to you, @Iain and @garagecoder! I might also look into the segmentation-based one, but for now opposed seems good enough for most situations (still testing though…). So, thanks!!

[hannoschwalm (Jens-Hanno Schwalm)]

If you run into “something” let us know! For me personally I like that the same code works so good for xtrans and bayers.

[agriggio (Alberto)]

Actually I’m using the RGB variant… (because in ART – inherited from RT – highlight recovery happens after demosaic, and I didn’t want to make a big change just to test out the new method.) So far, the RGB mode seems to work fine. There are some minor differences when I compare with dt, but overall they are quite similar.

[Iain]

No highlight recovery algorithms just colour adjustments and masks. Primarily using RGB levels.

2017-10-15_13-50-59-DSC_0026_05.NEF.xmp (21.3 KB)

[Terry (Terry Pinfold)]

@garagecoder

I did some test of the inpaint opposed algorithm on each area individually and the recovery was fine.

I notice highlight recovery module is one of the few modules that multiple instances is greyed out. I presume there is good reason for this omission that can not be overcome because of HLR’s position in the pipeline. It sounds like different localized HLR techniques being applied in a single image may have advantages at times if multiple instances was practical.

[Terry (Terry Pinfold)]

@hannoschwalm I just downloaded this image and used the segmentation based HLR method. All I needed to do was lower the clipping threshold until the magenta sky turned a beautiful blue. There is also another shot which I use to test HLR problems with and the segmentation based HLR method did a fantastic job once I lowered the clipping threshold on that image too. Great work making this method. It looks really promising and simple to use. Thanks for your hard work, it is appreciated.