Highlight recovery artifacts

Software darktable
1

This morning I was editing some very high-contrast images taken recently with my D750, when I noticed some really strange artifacts when using anything other than the default ‘Clip highlights’ method of the ‘Highlight recovery’ module. In particular the ‘Reconstruct color’ behaved unlike I have seen before.
I am wondering if this would be expected behavior, considering the overexposure from the sun. I thought I had really good results with the color reconstruction in the past, so I tried if it may be some bug in DT. Unfortunately, the behavior is exactly the same in 2.2.5 (now using 2.4.1).

200% crop with ‘Clip highlights’ - nothing special

HighContrast_ClipHighlights
HighContrast_ClipHighlights1277×1126 207 KB

200% crop with ‘Reconstruct LCh’ - jagged edges on the dark middle section

HighContrast_ReconstructLCh
HighContrast_ReconstructLCh1277×1127 209 KB

200% crop with ‘Reconstruct color’ - glitchy stuff at the edges of the overexposed area

HighContrast_ReconstructColor
HighContrast_ReconstructColor1277×1126 217 KB

200% crop showing raw highlights

HighContrast_RawHighlight
HighContrast_RawHighlight1277×1126 304 KB

And the original RAW, in case you are interested to try yourself.
_DSC2883.NEF (27.5 MB)

I hope anyone can shed some additional light on this issue (pun intended) :slight_smile:

2

Thought I’d see what RawTherapee did with this. I can’t make it do what Darktable is doing by changing recovery mode or demosaic method. Crop at full resolution with colour propagation for the h.reconstruction -
_DSC2883-Tree.jpg.out.pp3 (10.5 KB)
_DSC2883-Tree

3

Just did a bit of messing with your NEF using the libraw/dcraw processing, here’s what I see:

  1. When I just loaded the raw data with no processing, the exposure of the sun clips at the sensor limit, but you probably already know that…

  2. I then tried opening with different settings of the libraw/dcraw highlight paramter, defined thusly:

highlight=0|1|2|3+ - deal with image highlights, clip=0, 1=unclip, 2=blend, 3+=rebuild. dcraw: -H [0-9]

I can’t post any screenshots right now, but here’s what I saw:
0: same as what you see
1: magenta discoloration associated with letting the unclipped/out-of-bound data wrap back around to 0
2: A kind of smooth reddish-to-bluish transition into the clipped region, looks like the intuitive outcome of a blend. I think it looks quite nice… :smiley:
3: A definite, unblended greenish band in the region transiting from data-to-clip. I just did 3, I didn’t try any higher numbers.

Now, I don’t know specifically what dcraw is doing to “reconstruct”, but my simplistic thinking is this: ‘reconstruction’ implies using the data that went out-of-range to make something representative of it “in-range” for that region of the image. I think that’s an unreasonable expectation for data that’s just not available to start with. So, any reconstruction algorithm is probably going to yield sub-optimal results. Blending and clipping provide the most accurate rendition, IMHO, because 1) the light source is presenting a similar arrangement of tones, and 2) blending isn’t trying to do anything with data that isn’t there. In this case, clipping is really just recognizing the right-hand limit of the ‘blend’ the sun is presenting naturally.

Soooo, I’d say what you were looking at previously was some form of blend/clip. And in this case, blend or clip is really what you’d want to do with this data.

FWIW.

1 Like
4

With the Windows version, I can reproduce the LCh jagged edges, but not the Reconstruct color glitchy stuff.

Edit: some thing on Ubuntu. dt 2.4.1 in both cases.

5

Turning on lens correction removes most of it for me. So, I guess it is color fringing, which is quite typical for such hard contrast edges. Reconstruct LCh pronounces them I guess.

Glitchy stuff appears since the mode was added to the module. Changing the demosaicing method to VNG makes it a bit better. Anyway, the range in which the mode works is small.

Most of the highlights are totally clipped, as your last screenshot shows. In this case turning on the color reconstruction works quite well for me.

  • Demosaicing set to VNG with color smoothing 5x, match greens full and average mean
  • Module Highlight reconstruction with setting color reconstruction
  • Exposure -0.3
  • Lens correction on
  • Module Color Reconstruction on with default settings
  • White balance increased to ~6500K for eye candy

_DSC2883
_DSC28831156×733 272 KB

_DSC2883.NEF.xmp (5.4 KB)

2 Likes
6

It seems like RawTherapee and @ggbutcher’s experiments are at least capable of creating a smooth gradient going from the inner ring with all pixels blown out, to the outer ring with only the reds blown out. It does seem surprising that darktable fails here… Are the inpainting methods very different?

As the issue had me intrigued, I tried to reproduce it in other overexposed images. And I am now really confused and somehow convinced that something fishy is going on with the ‘Reconstruct color’ method. Take this _DSC2731.NEF (25.3 MB) together with this _DSC2731.NEF.xmp (7.2 KB) for example. I exaggerate the colors by pulling the base curve way down.

If I use ‘Clip highlights’ and change the threshold, I have a smooth transition going from dull and grey to magenta highlights. This is completely as I would expect.

highlight_clip
highlight_clip1199×741 1.93 MB

If I use ‘Reconstruct in LCh’ and again change the threshold, I get the following. Also what I would expect, and importantly, a very gradual transition into the magenta highlighting.

highlight_reconstructLCh
highlight_reconstructLCh1199×741 2.55 MB

However, for ‘Reconstruct color’ the behaviour of the threshold slider is very much non-linear. Basically, every value of 1,080 or more gives me this result.

_DSC2731_DT_01
_DSC2731_DT_011199×741 824 KB

Whereas even a slight decrease to 1,070 gives me this.

_DSC2731_DT
_DSC2731_DT1199×741 808 KB

Is this to be expected? Also, the artifacts are always present. Maybe minor in this over-the-top case, but still… This is the situation with the default value of 1,00:

_DSC2731_DT_02
_DSC2731_DT_021199×741 678 KB

The artifacts become rather pronounced when you decrease the threshold, but maybe that’s why the tooltip says that I “shouldn’t ever need to touch this”?

_DSC2731_DT_03
_DSC2731_DT_031199×741 829 KB

Could someone tell me if this is all expected behavior? And if so, why the artifacts show up? You can hit me with some source code explanation if you like.

Edit: oh and like @pk5dark says, the method of demosaicing really makes a difference here. VNG4 smoothes things out much more. All above images are with default PPG demosaicing. Amaze give dots instead of maze artifacts.

1 Like
7

I can’t say that it’s the expected behaviour but it is the behaviour I’m seeing. Highlight recovery, as far as I can tell, is not a strong point of darktable. Rawtherapee is much better.

10

I’ve had similar problems which may be related to a problem described in an open bug report.
_redmine.darktable.org - Bug #10283: new “reconstruct color” in “highlight recovery” producing weird artefacts outside highlights
One thing I have noticed is that removing the ‘green’ component in the white balance settings makes the problem disappear but of course that’s not much use in practice. 2016-11-26_IMG_2178_dt_02|690x440

2016-11-26_IMG_2178_dt_01
2016-11-26_IMG_2178_dt_011438×919 1.12 MB

11

Oh thanks, so there actually was a bug report about it! It hasn’t received much traction though. Dare I try to ask for a comment from @LebedevRI?
(this is the link to the redmine issue https://redmine.darktable.org/issues/10283 )

12

I guess you dared to.

(What was the question that i was supposed to elaborate on?)

13

I would hope that you can tell me that what I am seeing is the expected behavior of the “reconstruct color” method, or that it actually should do something else, but fails to do so. As I tried to indicate with examples above, it seems that something is off making the method unusable / unsuitable. Apparently, the maze-like artifacts that appear have also been filed as a bug report years ago.

Interestingly for example RawTherapee seems very capable to actually reconstruct some of the color in the overexposed areas, so such a thing could be possible.

14

Well, “reconstruct color” method is, and has been, broken from the start. So both i guess?
“reconstruct lch” is the best you can do in dt currently as far i’m aware.
And there is “color reconstruction” module, but it is color, not highlights.

Yep, absolutely. They have proper highlight inpainting, much like what dt’s “color reconstruction” does,
but for the actual highlights. It’s super powerful.

1 Like
15

I see, thanks for the quick response! I guess I have to deal with my overexposure in the LCh way for now :slight_smile:

Is there a wish to fix the behavior in the near future? Otherwise, it seems a bit useless to have a broken ‘feature’ in the software…

16

Or, you know, don’t overexpose :slight_smile:
raw overexposure indicator will tell you when you should have used less exposure next time :smiley:

Can’t really comment on the rest.