I was exploring this here…
FYI
1 Like
Rawspeed has a single value of 13653 for the Canon 50D
RawTherapee has different values base on the ISO. For the 320 ISO, it uses 12700.
All 3 are different (ExifTool, Rawspeed and RawTherapee).
Combined with the test Kofa did, I would use 12700 on this image.
OP, for other ISOs, this is what RawTherapee has. You can create Auto Presets in dt to adjust the white point based on these.
“white”: [
{ “iso”: [ 100, 125 ], “levels”: 13300 }, // typical 13432
{ “iso”: [ 160, 320, 640, 1250 ], “levels”: 12700 }, // typical 12790-12810
{ “iso”: [ 200, 250, 400, 500, 800, 1000, 1600, 2000, 2500, 3200 ], “levels”: 15630 }, // typical 15763-15733
{ “iso”: [ 6400, 12800 ], “levels”: 16200 } // typical 16383
- Taken from RawTherapee GitHub - Beep6581/RawTherapee: A powerful cross-platform raw photo processing program at commit
7b51a07
1 Like
Tried to create contrast by colors - maybe the result is a bit too colorful, but I like it.
_MG_0814.cr2.xmp (46.4 KB)
Thanks for sharing this nice image!
1 Like
Fujifilm? Wasn’t this taken with a Canon 50D?
I was trying to reference the commit I used.
I did an edit to avoid confusion. The values are for the 50D camera.
1 Like
Today I learnt something new about DT. I opened this image in DT and had the infamous magenta highlights in the clouds, I turned on the raw clipping detection option, adjusted the white point value until the magenta areas were revealed as clipped. I turned off the raw clipping detection and the image looked great. What a simple fix.
I then tried the method on another picture with very bad magenta cloud issue. Again the method worked by just lowering the white point until the magenta clouds disappeared. However, this new image needed further highlight reconstruction to restore the blue sky and cloud detail. I experimented with all the options available in the current Windows weekly build. Only the inpaint opposed and the segmentation based method worked well.
Thanks for the hint about changing the white point. It is easy to adjust and needs to become part of my go to fix if I see magenta highlights.
1 Like
I think you can do this but really I don’t think it’s a routine editing tactic. If the value is correct as expected by the camera thing are generally fine. In DT these things have recently been looked at and addressed as the move to scene referred and modern wb didn’t really play nice with in particular the default clip highlight mode. In many cases I find looking at all these play raw files in older versions that using the correct white point and legacy wb was enough to make thongs behave. Now with inpaint and segmentation it’s even better
Thank you! Separate from this issue, I had been trying to figure out why the raw overexposure toggles stopped working; I know they were working fine in a previous version, but after upgrading to 4.0 they stopped displaying anything even when I knew an area was clipped. Any ideas on why the white point is being incorrectly set in raw black/white point? @g-man thanks for the ISO values from Raw Therapee - I’ll try making auto-presets for those.
After setting white point to 12800, even just using highlight reconstruction with the reconstruct color mode on 4.0.1 looks okay (not great, but okay):
Agreed, turning off color calibration and setting white balance to as shot and then using highlight reconstruction in clip highlights mode seems to work pretty well:
Is color calibration really not compatible with highlight reconstruction (or apparently filmic’s reconstruction), at least prior to 4.2?
Once inpaint opposed is available with 4.2, will there ever be a reason to use the other values in highlight reconstruction?
If you watch APs video on HLR with GLP around the 5 min window of 25 to 30 min he explains it… basically the way I understand it is that the old methods for HLR rely on the wb done early as it was before CC was introduced. which now happens later in the pipeline than the wb module So using these methods is less problematic with legacy wb.as they were designed to work with that. In 4.2 you have new modes that allow for the CAT wb you get with CC module but if using versions without inpaint and segmentation and you want to use CC module wb then filmic HLR with or without GLPHLR is going to handle highlights better The issue is to some extent I think also impacted by some math in Filmic v6 that doesn’t desaturated the same way as v5 did and does some gamut mapping… There are several good threads here over the last few months hashing through all this. Finally the addition of the sigmoid tone mapper also gives more options in a simplified module so I think people will like 4.2
The cause of the blue clouds is the colour corrections done for white balancing and where they are done: a first correction is done in the white balance module, the ‘traditional’ modes in highlight correction use that to decide how “white” looks. Then later you get the color calibration which changes all colours depending on the new white balance setting (if different from 5000 K daylight); that change also affects the white generated by those highlight reconstruction modes…
One trick that sometimes works is using two copies of color calibration with a mask on the clipped regions: one copy sets WB to pipeline, the other to what it should be for the rest of the image. At least your clouds will be white that way. Ignore hte warning about WB being applied twice, though.
Or don’t use the white balance functions of color calibration, instead white balance the old way…
1 Like
The highlights reconstruction module and filmics reconstruction work on totally different data, the first basically takes raw data and “corrects” them acccording to the clip threshold, the second takes rgb data late in pipeline and corrects according to close or above white point.
Exactly.
Just one more comment here, if you evaluate the white point of your images and suspect a wrong setting the raw indicator might not be the best way to check. Why? The different HLR algorithms use slightly modified clipping thresholds, the raw indicator does not know about this - the mask button in HLR does.
1 Like
Don’t the LHR algos derive their clipping thresholds from the real sensor clipping threshold? If it is so, then I still think getting the raw clipping threshold correct is important.
And one more question: why is it that I see more images recently where darktable sets too high a raw white level? What has changed?
I don’t think any has changed in the raw white point in dt. Looking back at rawspeed commits, it being like this for a while. We probably blamed filmic or using legacy white balance masked the problem or something else in the past, but now we know.
I’ve been doing more reading in the subject and it can get complex since not all channels clip at the same point in the sensor. Further, some might start to behave not so linear (but have yet to clip).
Having a raw histogram by channel available in the raw black/white module like rawdigger has could be helpful.
The “real sensor clipping threshold” is corrected via white point in rawprepare so the clipping is normalized to 1.0, (everything above is clipped). The HLR uses the normalized data (in reality also corrected by temperature)
Does raw clipping not show the pixels that have values reaching (or, if the threshold is manually lowered too much, exceed) the raw white value? Because, if I understand correctly, the ‘magenta highlights’ phenomenon is caused by green being blown, ‘not being able to keep up with’ red and blue as the scene gets brighter. Then, the white balance multipliers transform the green-clipped values into magenta, instead of neutral. However, on this image, and many others, magenta areas are not shown as clipped, and when I lower the raw white level, there is a certain threshold, when a large number of pixels gets shown as clipped.
For the following screenshots, only raw black/white point, white balance (camera reference), demosaic, input color profile and output color profile were enabled.
raw white = 12812:
12811:
darktable’s default for this camera is 13653. As @g-man noted above, RT’s ISO 320 values are a good match with what I found:
{ “iso”: [ 160, 320, 640, 1250 ], “levels”: 12700 }, // typical 12790-12810I am just curious and throwing this out… Some time ago and maybe in a few other videos AP demonstrated using CC because you can mask it and also for difficult and weird lighting but is there a massive advantage if you are just shooting daylight landscapes or simple outdoor shots to using this… I guess the issues are handled by the new HLR modes if they pop up but Just out of curiosity is there some major advantage to using it??
1 Like
@kofa I think we should move this conv to a new thread. This article is interesting to the subject. Look at figure 24.
1 Like
This is a draft HLR doc for dt written by @Iain maybe helping …
Darktable new highlight reconstruction method user documentation.zip (45.5 KB)