Help with highlight reconstruction of clouds

Here are two versions, one with the new HLR and Sigmoid, and one more suitable for darktable 4.0.1 with FilmicRGB and the old HLR.

4.1.0, Sigmoid:


_MG_0814 Sigmoid.cr2.xmp (13.8 KB)

and 4.0.1, FilmicRGB

_MG_0814_FilmicRGB.cr2.xmp (16.1 KB)

Notice that the xmp with the Sigmoid version will probably not open in darktable 4.0.1 or older.
Other than tone mapping and HLR, they are more or less the same.

1 Like

Since it is on master, it will be part of 4.2. The only scenario for it not to make it would be q revert because someone finds a major issue. That’s extremely unlikely at this point since we have been using it for a few months now. Because how much better it is over the current default, it will be the default in 4.2.

1 Like

The while level for your camera seems to be wrong. Turn off highlight reconstruction, turn on raw clipping detection, and lower the white point until the magenta is marked as clipped.

I found that 12800 is a good value.

Having the wrong raw clipping value impacts all highlight recovery methods.

If you are on darktable 4.0 or older, try highlight reconstruction with recover in LCh (but disable color calibration and set white balance to as shot):

Or, if you are on at least 4.0, you can try keeping white balance as camera reference and keep color calibration enabled, and use guided laplacians as your highlight reconstruction method (but this time, it did not work for me).

If you are on the development branch, use inpaint opposed as the recovery method (it is the new default):

10 Likes

Good catch on the white balance. I think it is the first time I see a wrong white point on a cr2 file.

Exif values for this file are not that low but whatever works…

image

What software … art has AGX luts but nothing in DT or RT right… Blender??? Other??

1 Like

Great answer…even with the default white level …things are fine as long as you are using legacy wb and not clip highlights with the modern wb ie color calibration… Going to legacy wb and a touch of tone eq totally restores that small segment and no blue/cyan stuff…

The work done with the new HLR to make it compatible and mostly free of this sort of things is nice…

Any OCIOv2 software (you can tell I’m a fan). Blender works. I use Olive! (Open source NLE video editor)

In Darktable you can use a LUT, but it’s a LUT… results may vary.

(Looking at ART right now, thanks for the heads-up!)

My version… ( filmique rvb ).

_MG_0814.cr2.xmp (21.1 KB)

4 Likes

I was exploring this here…

FYI

1 Like

Hi @garibaldi,

here is my edit :slightly_smiling_face:


_MG_0814_02.cr2.xmp (19.6 KB)

1 Like

Rawspeed has a single value of 13653 for the Canon 50D
RawTherapee has different values base on the ISO. For the 320 ISO, it uses 12700.
All 3 are different (ExifTool, Rawspeed and RawTherapee).

Combined with the test Kofa did, I would use 12700 on this image.

OP, for other ISOs, this is what RawTherapee has. You can create Auto Presets in dt to adjust the white point based on these.

“white”: [
{ “iso”: [ 100, 125 ], “levels”: 13300 }, // typical 13432
{ “iso”: [ 160, 320, 640, 1250 ], “levels”: 12700 }, // typical 12790-12810
{ “iso”: [ 200, 250, 400, 500, 800, 1000, 1600, 2000, 2500, 3200 ], “levels”: 15630 }, // typical 15763-15733
{ “iso”: [ 6400, 12800 ], “levels”: 16200 } // typical 16383

1 Like

Tried to create contrast by colors - maybe the result is a bit too colorful, but I like it.


_MG_0814.cr2.xmp (46.4 KB)

Thanks for sharing this nice image!

1 Like

Fujifilm? Wasn’t this taken with a Canon 50D?

I was trying to reference the commit I used.

I did an edit to avoid confusion. The values are for the 50D camera.

1 Like

Today I learnt something new about DT. I opened this image in DT and had the infamous magenta highlights in the clouds, I turned on the raw clipping detection option, adjusted the white point value until the magenta areas were revealed as clipped. I turned off the raw clipping detection and the image looked great. What a simple fix.

I then tried the method on another picture with very bad magenta cloud issue. Again the method worked by just lowering the white point until the magenta clouds disappeared. However, this new image needed further highlight reconstruction to restore the blue sky and cloud detail. I experimented with all the options available in the current Windows weekly build. Only the inpaint opposed and the segmentation based method worked well.

Thanks for the hint about changing the white point. It is easy to adjust and needs to become part of my go to fix if I see magenta highlights.

1 Like

I think you can do this but really I don’t think it’s a routine editing tactic. If the value is correct as expected by the camera thing are generally fine. In DT these things have recently been looked at and addressed as the move to scene referred and modern wb didn’t really play nice with in particular the default clip highlight mode. In many cases I find looking at all these play raw files in older versions that using the correct white point and legacy wb was enough to make thongs behave. Now with inpaint and segmentation it’s even better

Thank you! Separate from this issue, I had been trying to figure out why the raw overexposure toggles stopped working; I know they were working fine in a previous version, but after upgrading to 4.0 they stopped displaying anything even when I knew an area was clipped. Any ideas on why the white point is being incorrectly set in raw black/white point? @g-man thanks for the ISO values from Raw Therapee - I’ll try making auto-presets for those.

After setting white point to 12800, even just using highlight reconstruction with the reconstruct color mode on 4.0.1 looks okay (not great, but okay):

Agreed, turning off color calibration and setting white balance to as shot and then using highlight reconstruction in clip highlights mode seems to work pretty well:

Is color calibration really not compatible with highlight reconstruction (or apparently filmic’s reconstruction), at least prior to 4.2?

Once inpaint opposed is available with 4.2, will there ever be a reason to use the other values in highlight reconstruction?

If you watch APs video on HLR with GLP around the 5 min window of 25 to 30 min he explains it… basically the way I understand it is that the old methods for HLR rely on the wb done early as it was before CC was introduced. which now happens later in the pipeline than the wb module So using these methods is less problematic with legacy wb.as they were designed to work with that. In 4.2 you have new modes that allow for the CAT wb you get with CC module but if using versions without inpaint and segmentation and you want to use CC module wb then filmic HLR with or without GLPHLR is going to handle highlights better The issue is to some extent I think also impacted by some math in Filmic v6 that doesn’t desaturated the same way as v5 did and does some gamut mapping… There are several good threads here over the last few months hashing through all this. Finally the addition of the sigmoid tone mapper also gives more options in a simplified module so I think people will like 4.2

The cause of the blue clouds is the colour corrections done for white balancing and where they are done: a first correction is done in the white balance module, the ‘traditional’ modes in highlight correction use that to decide how “white” looks. Then later you get the color calibration which changes all colours depending on the new white balance setting (if different from 5000 K daylight); that change also affects the white generated by those highlight reconstruction modes…

One trick that sometimes works is using two copies of color calibration with a mask on the clipped regions: one copy sets WB to pipeline, the other to what it should be for the rest of the image. At least your clouds will be white that way. Ignore hte warning about WB being applied twice, though.
Or don’t use the white balance functions of color calibration, instead white balance the old way…

1 Like