Help with highlight reconstruction of clouds

Exif values for this file are not that low but whatever works…

image

What software … art has AGX luts but nothing in DT or RT right… Blender??? Other??

1 Like

Great answer…even with the default white level …things are fine as long as you are using legacy wb and not clip highlights with the modern wb ie color calibration… Going to legacy wb and a touch of tone eq totally restores that small segment and no blue/cyan stuff…

The work done with the new HLR to make it compatible and mostly free of this sort of things is nice…

Any OCIOv2 software (you can tell I’m a fan). Blender works. I use Olive! (Open source NLE video editor)

In Darktable you can use a LUT, but it’s a LUT… results may vary.

(Looking at ART right now, thanks for the heads-up!)

My version… ( filmique rvb ).

_MG_0814.cr2.xmp (21.1 KB)

4 Likes

I was exploring this here…

FYI

1 Like

Hi @garibaldi,

here is my edit :slightly_smiling_face:


_MG_0814_02.cr2.xmp (19.6 KB)

1 Like

Rawspeed has a single value of 13653 for the Canon 50D
RawTherapee has different values base on the ISO. For the 320 ISO, it uses 12700.
All 3 are different (ExifTool, Rawspeed and RawTherapee).

Combined with the test Kofa did, I would use 12700 on this image.

OP, for other ISOs, this is what RawTherapee has. You can create Auto Presets in dt to adjust the white point based on these.

“white”: [
{ “iso”: [ 100, 125 ], “levels”: 13300 }, // typical 13432
{ “iso”: [ 160, 320, 640, 1250 ], “levels”: 12700 }, // typical 12790-12810
{ “iso”: [ 200, 250, 400, 500, 800, 1000, 1600, 2000, 2500, 3200 ], “levels”: 15630 }, // typical 15763-15733
{ “iso”: [ 6400, 12800 ], “levels”: 16200 } // typical 16383

1 Like

Tried to create contrast by colors - maybe the result is a bit too colorful, but I like it.


_MG_0814.cr2.xmp (46.4 KB)

Thanks for sharing this nice image!

1 Like

Fujifilm? Wasn’t this taken with a Canon 50D?

I was trying to reference the commit I used.

I did an edit to avoid confusion. The values are for the 50D camera.

1 Like

Today I learnt something new about DT. I opened this image in DT and had the infamous magenta highlights in the clouds, I turned on the raw clipping detection option, adjusted the white point value until the magenta areas were revealed as clipped. I turned off the raw clipping detection and the image looked great. What a simple fix.

I then tried the method on another picture with very bad magenta cloud issue. Again the method worked by just lowering the white point until the magenta clouds disappeared. However, this new image needed further highlight reconstruction to restore the blue sky and cloud detail. I experimented with all the options available in the current Windows weekly build. Only the inpaint opposed and the segmentation based method worked well.

Thanks for the hint about changing the white point. It is easy to adjust and needs to become part of my go to fix if I see magenta highlights.

1 Like

I think you can do this but really I don’t think it’s a routine editing tactic. If the value is correct as expected by the camera thing are generally fine. In DT these things have recently been looked at and addressed as the move to scene referred and modern wb didn’t really play nice with in particular the default clip highlight mode. In many cases I find looking at all these play raw files in older versions that using the correct white point and legacy wb was enough to make thongs behave. Now with inpaint and segmentation it’s even better

Thank you! Separate from this issue, I had been trying to figure out why the raw overexposure toggles stopped working; I know they were working fine in a previous version, but after upgrading to 4.0 they stopped displaying anything even when I knew an area was clipped. Any ideas on why the white point is being incorrectly set in raw black/white point? @g-man thanks for the ISO values from Raw Therapee - I’ll try making auto-presets for those.

After setting white point to 12800, even just using highlight reconstruction with the reconstruct color mode on 4.0.1 looks okay (not great, but okay):

Agreed, turning off color calibration and setting white balance to as shot and then using highlight reconstruction in clip highlights mode seems to work pretty well:

Is color calibration really not compatible with highlight reconstruction (or apparently filmic’s reconstruction), at least prior to 4.2?

Once inpaint opposed is available with 4.2, will there ever be a reason to use the other values in highlight reconstruction?

If you watch APs video on HLR with GLP around the 5 min window of 25 to 30 min he explains it… basically the way I understand it is that the old methods for HLR rely on the wb done early as it was before CC was introduced. which now happens later in the pipeline than the wb module So using these methods is less problematic with legacy wb.as they were designed to work with that. In 4.2 you have new modes that allow for the CAT wb you get with CC module but if using versions without inpaint and segmentation and you want to use CC module wb then filmic HLR with or without GLPHLR is going to handle highlights better The issue is to some extent I think also impacted by some math in Filmic v6 that doesn’t desaturated the same way as v5 did and does some gamut mapping… There are several good threads here over the last few months hashing through all this. Finally the addition of the sigmoid tone mapper also gives more options in a simplified module so I think people will like 4.2

The cause of the blue clouds is the colour corrections done for white balancing and where they are done: a first correction is done in the white balance module, the ‘traditional’ modes in highlight correction use that to decide how “white” looks. Then later you get the color calibration which changes all colours depending on the new white balance setting (if different from 5000 K daylight); that change also affects the white generated by those highlight reconstruction modes…

One trick that sometimes works is using two copies of color calibration with a mask on the clipped regions: one copy sets WB to pipeline, the other to what it should be for the rest of the image. At least your clouds will be white that way. Ignore hte warning about WB being applied twice, though.
Or don’t use the white balance functions of color calibration, instead white balance the old way…

1 Like

The highlights reconstruction module and filmics reconstruction work on totally different data, the first basically takes raw data and “corrects” them acccording to the clip threshold, the second takes rgb data late in pipeline and corrects according to close or above white point.

Exactly.

Just one more comment here, if you evaluate the white point of your images and suspect a wrong setting the raw indicator might not be the best way to check. Why? The different HLR algorithms use slightly modified clipping thresholds, the raw indicator does not know about this - the mask button in HLR does.

1 Like

Don’t the LHR algos derive their clipping thresholds from the real sensor clipping threshold? If it is so, then I still think getting the raw clipping threshold correct is important.

And one more question: why is it that I see more images recently where darktable sets too high a raw white level? What has changed?

I don’t think any has changed in the raw white point in dt. Looking back at rawspeed commits, it being like this for a while. We probably blamed filmic or using legacy white balance masked the problem or something else in the past, but now we know.

I’ve been doing more reading in the subject and it can get complex since not all channels clip at the same point in the sensor. Further, some might start to behave not so linear (but have yet to clip).

Having a raw histogram by channel available in the raw black/white module like rawdigger has could be helpful.

The “real sensor clipping threshold” is corrected via white point in rawprepare so the clipping is normalized to 1.0, (everything above is clipped). The HLR uses the normalized data (in reality also corrected by temperature)