This is my result from RT 5.8. No extra highlight reconstruction procedure was needed here. To begin by using the DRC (Dynamic Range Compression) with the default setting does the highlighting job. Then some slight Lab* up to the taste tuning, Graduated Filter for the sky, slight Vibrance, and voila!
3 Likes
Ya np…you can make one using the icc tool in RT…takes no time but its not likely bothering you…
Hi All. I have decided to share my processing steps for this image. I am hoping that some people new to DT may find it helpful. I will included the xmp file as well so you can check out my masking.
In my version I have less blue sky region than some of the versions posted here. I suspect the true blue sky region is not clipped in this image, but clouds are. I suspect that in recreating the highlights some algorithms could be creating more blue sky than was present in the scene. Maybe only the photographer can be sure about this.
Magenta Highlights Problem
- Open image and there is an extreme magenta problem in the highlights
- In the highlight reconstruction module
a. Change method to reconstruct in LCh to get back the detail.
b. To get rid of the magenta in the clouds lower the clipping threshold, but don’t go too far or the saturation in the blue sky would be lost. Maybe try 0.763. - With the exposure module create a drawn and parametric mask to select just the sky.
a. Draw a rough path around the sky, but carefully exclude the small section of the most distant mountain.
b. Create a parametric mask while displaying the mask. Use the g slider values of 14.5, 14.5, 100, 100 to isolate the sky
c. Hide the mask and set exposure to -0.36 to darken the sky
d. Apply a small amount of feather and blur radius for the mask to ensure a invisible transition. Maybe 20px for feather and 10 px for blur, but this may vary depending on you individually created mask. - Create a duplicate instance of this exposure module. Being a duplicate means the previously created mask is included in the new instance.
a. Reveal the mask
b. Invert all channels polarity for the parametric mask
c. Tweak the parametric mask to avoid masking the dark cloud. Changing the 14.5 values to about 12.2 should be sufficient.
d. Hide the mask and set the exposure value to +1.3
e. Increase the blur radius of the mask to about 60 px to avoid the halo artefacts - In the filmic scene tab select auto tune levels
- Apply denoise, sharpen and add basic colourfulness (this is a useful style to save)
- Add local contrast at default values
- Add shadow and highlights using bilateral filter for softening method
- Use tone equalizer to slightly brighten the darkest trees in the foreground. The tone equalizer mask will mask will require some adjustment of the exposure compensation.
- Go to the color zones module and increase the saturation for the blue sky and decrease the magenta saturation to remove any magenta tint remaining in the clouds.
- Take a snapshot
- Go to duplicate manager and create a duplicate (this includes all edits)
- With the duplicate now active return to the first exposure module in the pipeline and change the sky exposure to 0.0 EV.
- Use the snapshot to compare the two versions and decide which sky exposure you prefer.
Magenta Highlights.cr2.xmp (15.3 KB)
2 Likes
@Terry, that’s a nice walkthrough. I think that patch of sky should be blue, but agree that without having been there it’s hard to be sure!
Here’s my new version using the new highlight recovery mode, in Todd’s build. (thanks!)
I’m quite impressed with the new mode. I haven’t quite got the hang of the adjustments yet, but a bit of trial and error works well. It’s a lot quicker than the guided laplacian mode which is great for me.I think it immediately looks a lot better in terms of the large patch of blue (if it is), but I wasn’t able to entirely get rid of a reddish cast in the areas where the blue channel (I think?) is clipped. It’s still impressive though.
I’m using filmic in V6 and max rgb - using a mode that desaturates gets rid of the reddish bits, but also affects the blue sky - so instead I used color balance rgb with a parametric mask in the jz and hz channels to desaturate only the reddish highlights.
Apart from that I used local contrast (on the highlights), diffuse and sharpen, an exposure gradient on the sky and also tone eq to pull the highlights down a bit.
I don’t think I’m being as clear as Terry 
I’m not entirely happy this version - it doesn’t look quite natural to me, but the sky is good I think!
Uploading: 2014-05-30_19-47-01.cr2.xmp…
3 Likes
Your version looks good. I want to try Todd’s build soon. Thanks Todd and all the developers for the continued work in DT. I love how there can be so many different approaches to tackling problems or interpreting scenes in DT.
For this shot I really would have liked to have tackled the problem in the camera by having one exposure for the sky and one exposure for the foreground. But that is not always possible. However, with exposure bracketing and either a tripod or suitable alignment software the issue is usually best tackled in the camera and not the editing. Gimp and layer masks often work wonders with that and dare I say it Lightroom which will align hand-held images. One of the few reasons I occasionally use Lightroom.
One of the rare occasions when I think it might be nice to have Lightroom… 
TBH though I don’t often want to do this. Maybe I should try a bit more experimentation.
I echo this. It’s really great.
You are right, with your opinion, that this problem should have been tackled already in the camera. But the problem in this case was me. In this situation, I should have exposed a little bit darker. That would have been enough. Even with my old 550D and it’s quite limited dynamic range. No exposure bracketing needed. 
The problem with darker exposures and lifting the shadows is excessive noise. Really excessive noise. But yes, more modern sensors have a better dynamic range. Here is an example I use with my students. The image is a crop from a Canon G16 compact camera. Exposures have been matched in the editing of the RAW files. The Histograms represent the RAW file prior to matching.
Thanks Todd. I will give this build a whirl.
1 Like
Of course, shadow lifting is causing some noise. But on that image there isn’t much noise, so it would have been enough to just expose 1 stop darker. I absolutely agree with you, that this isn’t possible with any picture, especially not, when you use high ISO.
I did notice that noise was not an issue on the image supplied here and lifting the shadows was not an issue. Maybe pushing the exposure a third or two thirds of a stop darker would have helped. But such a wide dynamic range to capture here regardless. It was an interesting exercise and a good example of the crazy magenta highlight issue that occasionally happens in DT.
And a good lesson to cope with it as well. I have never played with the white point before.
And the best thing is, that we were shown, that the dt-devs are heroes, working on all the issues which are left in dt. 
Not just in dt, dt makes it visible.
But the magenta in itself is a logical result of white balancing the image, where red and blue are scaled by factors 1>. So a clipped area (r==g==b==1 in the raw) will end up with g==1, r>1, b>1. r+b gives magenta.
That’s because you shouldn’t have to (play with the raw black and white points, that is): most cameras store the correct values in the “standard” EXIF data, where dt picks them up. In this case, the required values are stored in the (undocumented) “maker note” section of EXIF, which dt in general can’t decode.
(some parts of some “maker note” sections are understood, and can be used. But that’s due to patient testing by some developers, not thanks to any generosity from camera makers).
Well, you’re completely right about handling this in camera, and not in editing, as long as the use of alignment software, Gimp and layer masks, or Lightroom does not count as editing, but rather as in-camera solutions.
BTW, I never use a tripod, not even in the rare cases when I do exposure bracketing; Hugin (and align_image_stack) work just fine with handheld images (whether bracketed or for panoramas).
1 Like
RT’s camconst.json has this section for the 550D -
{ // Quality B, ISO and aperture WL data by … at RawTherapee forums, missing samples safely guessed
“make_model”: [ “Canon EOS 550D”, “Canon EOS Rebel T2i”, “Canon EOS Kiss X4” ],
“dcraw_matrix”: [ 6941,-1164,-857,-3825,11597,2534,-416,1540,6039 ], // dcraw 550d
“ranges”: {
“white”: [
{ “iso”: [ 100, 125 ], “levels”: 13480 }, // typical 13584
{ “iso”: [ 160, 320, 640, 1250, 2500 ], “levels”: 12550 }, // typical 12650
{ “iso”: [ 200, 250, 400, 500, 800, 1000, 1600, 2000, 3200, 4000, 5000, 6400, 12800 ], “levels”: 15200 } // typical 15304
],
and the pic is at ISO200, so the whitepoint used should be around 15200, assuming camconst is correct.
I’ve no idea about the makernote values.
NOTE ALL: If you’re interested in white points, you can check what your own camera is doing, camconst.json tells you how to do it. It mentions using Rawdigger but you can also use DCRAW to see where the white point is. On Windows years ago I did e.g.
dcraw -c -4 -D 427A0791.cr2 | C:"Program Files"\GnuWin32\bin\pgmhist.exe > outfile427A0791.txt
The first part outputs all the pixel values then “pgmhist” creates a histogram of them which gets written out. I can’t remember where pgmhist came from, but I’m guessing there’s some linux way to do this. Here’s a section from an output file -
|15267|182|0.000902%| 100%|
|15268|360|0.00269%| 100%|
|15269|166|0.00351%| 100%|
|15272|1|0.00352%| 100%|
|15282|5037568| 25%| 100%|
|15283|10074816| 74.9%| 75%|
|15284|5037731| 99.9%| 25.1%|
|15286|46| 99.9%|0.0967%|
|15287|45| 99.9%|0.0964%|
|15325|37| 99.9%|0.0962%|
|15326|34| 99.9%|0.096%|
|15330|4747| 99.9%|0.0959%|
|15331|9894| 100%|0.0723%|
|15332|4678| 100%|0.0233%|
|15334|1| 100%|7.44e-005%|
|15415|1| 100%|6.94e-005%|
|15430|1| 100%|6.45e-005%|
First column is the pixel value.
2nd is the no. of pixels with that value.
3rd and 4th are cumulative pixel count percents going up and down.
The white point is pretty obvious at 15283 + or - one. Values 15282-4 account for nearly all the 20 megapixels of the Canon 6D sensor.
camconst.txt (154.1 KB)
(2 years old)
2 Likes
Yeah. For me the problem is that we use const data, here depending on ISO. The ISO dependency can be found on many / most cameras.
I see the table but exiftool tells something different. BTW i would trust exiftool 
15200 is simply wrong for the image in question.
Is it not possible to add an auto -detect button, just in case the user suspects darktable’s default is wrong?
Something like what rawproc does, when @ggbutcher adds the Black/White Point tool to his stack, which has a data option (as well as camera).
1 Like
Seems so. Interesting that @kofa determined a value just one away from the typical ISO 100 white point!
?? do you mean dt doesn’t take ISO into account?
One of the more useful things I put in to rawproc, turns out. In my processing, I still use the exif black subtract, but I let the blackwhitepoint tool figure out the top end.
In darktable, such a thing might mess with the assumptions of the tone curves… ??