Thank you! Separate from this issue, I had been trying to figure out why the raw overexposure toggles stopped working; I know they were working fine in a previous version, but after upgrading to 4.0 they stopped displaying anything even when I knew an area was clipped. Any ideas on why the white point is being incorrectly set in raw black/white point? @g-man thanks for the ISO values from Raw Therapee - I’ll try making auto-presets for those.
After setting white point to 12800, even just using highlight reconstruction with the reconstruct color mode on 4.0.1 looks okay (not great, but okay):
Agreed, turning off color calibration and setting white balance to as shot and then using highlight reconstruction in clip highlights mode seems to work pretty well:
If you watch APs video on HLR with GLP around the 5 min window of 25 to 30 min he explains it… basically the way I understand it is that the old methods for HLR rely on the wb done early as it was before CC was introduced. which now happens later in the pipeline than the wb module So using these methods is less problematic with legacy wb.as they were designed to work with that. In 4.2 you have new modes that allow for the CAT wb you get with CC module but if using versions without inpaint and segmentation and you want to use CC module wb then filmic HLR with or without GLPHLR is going to handle highlights better The issue is to some extent I think also impacted by some math in Filmic v6 that doesn’t desaturated the same way as v5 did and does some gamut mapping… There are several good threads here over the last few months hashing through all this. Finally the addition of the sigmoid tone mapper also gives more options in a simplified module so I think people will like 4.2
The cause of the blue clouds is the colour corrections done for white balancing and where they are done: a first correction is done in the white balance module, the ‘traditional’ modes in highlight correction use that to decide how “white” looks. Then later you get the color calibration which changes all colours depending on the new white balance setting (if different from 5000 K daylight); that change also affects the white generated by those highlight reconstruction modes…
One trick that sometimes works is using two copies of color calibration with a mask on the clipped regions: one copy sets WB to pipeline, the other to what it should be for the rest of the image. At least your clouds will be white that way. Ignore hte warning about WB being applied twice, though.
Or don’t use the white balance functions of color calibration, instead white balance the old way…
The highlights reconstruction module and filmics reconstruction work on totally different data, the first basically takes raw data and “corrects” them acccording to the clip threshold, the second takes rgb data late in pipeline and corrects according to close or above white point.
Exactly.
Just one more comment here, if you evaluate the white point of your images and suspect a wrong setting the raw indicator might not be the best way to check. Why? The different HLR algorithms use slightly modified clipping thresholds, the raw indicator does not know about this - the mask button in HLR does.
Don’t the LHR algos derive their clipping thresholds from the real sensor clipping threshold? If it is so, then I still think getting the raw clipping threshold correct is important.
And one more question: why is it that I see more images recently where darktable sets too high a raw white level? What has changed?
I don’t think any has changed in the raw white point in dt. Looking back at rawspeed commits, it being like this for a while. We probably blamed filmic or using legacy white balance masked the problem or something else in the past, but now we know.
I’ve been doing more reading in the subject and it can get complex since not all channels clip at the same point in the sensor. Further, some might start to behave not so linear (but have yet to clip).
Having a raw histogram by channel available in the raw black/white module like rawdigger has could be helpful.
The “real sensor clipping threshold” is corrected via white point in rawprepare so the clipping is normalized to 1.0, (everything above is clipped). The HLR uses the normalized data (in reality also corrected by temperature)
Does raw clipping not show the pixels that have values reaching (or, if the threshold is manually lowered too much, exceed) the raw white value? Because, if I understand correctly, the ‘magenta highlights’ phenomenon is caused by green being blown, ‘not being able to keep up with’ red and blue as the scene gets brighter. Then, the white balance multipliers transform the green-clipped values into magenta, instead of neutral. However, on this image, and many others, magenta areas are not shown as clipped, and when I lower the raw white level, there is a certain threshold, when a large number of pixels gets shown as clipped.
For the following screenshots, only raw black/white point, white balance (camera reference), demosaic, input color profile and output color profile were enabled.
I am just curious and throwing this out… Some time ago and maybe in a few other videos AP demonstrated using CC because you can mask it and also for difficult and weird lighting but is there a massive advantage if you are just shooting daylight landscapes or simple outdoor shots to using this… I guess the issues are handled by the new HLR modes if they pop up but Just out of curiosity is there some major advantage to using it??
