Local adjustments - Color appearance (and log) are incompatible with highlight correction?

The two local adjustment tools “break” overexposed areas. Dramatically reducing scope makes no difference despite Preview delta E showing areas as unaffected.

First file from Please try to save this overexposed image in Darktable with Neutral applied but inpaint opposed activated.


_V3A0302.jpg.out.pp3 (14.0 KB)

Then local ciecam applied as full spot centered on the black clothes. Scope adjusted so that sky should be unaffected. See preview below/

Exported file with blocked out sky.


_V3A0302-1.jpg.out.pp3 (18.9 KB)

The issues is that almost all properly exposed photographs have some blown highlights. The above example is exaggerated but the same thing happens with reflections, lamp or sun in the frame but otherwise correct exposure.

This can be solved by adding exluding spots with high scope values on all blown out areas but this is a very slow and labour intensive solution. In addition the excluding spots tend to require high scope values to remove all yellow. So high that they affect even relatively dark areas that may overlap.

Is there a reason scope doesn’t affect these tools? Or is it a bug?

This was tested on Version: 5.9-718-gd1750e827 but the behaviour has always been like this I think.

@nosle
Thank you for your participation, and this does not only concern this topic :+1:

This image is extremely overexposed, almost unusable.

The remarks you make on the behavior of LA and this type of image are real, I will look more closely, but the documentation (Rawpedia) recommends the use of excluding spot.
https://rawpedia.rawtherapee.com/Local_Adjustments#Log_Encoding_and_Highlight_Recovery

However - regardless of LA’s behavior - there are ways to improve the situation a little:

  • in RAW-tab - move White-point correction to 0.40
  • in Exposure Tab - replace Inpaint Opposed by Color Propagation
  • the with LA what you want, but no miracle: Color Appearance (your choice) with Scope = 60, Lightness J = 6.7, Contrast J = 42.9

Of course we can arrange the sky with one or two RT-spots, to remove the purple and/or make it bluer with “LA Color correction grid”.

I don’t says it’s good, but acceptable

Jacques

The core of the question is why you can’t use scope to exclude blown highlights from these modules. You can, seemingly effectively, exclude other parts of the image, in other local modules, by using scope.

Is it that scope never fully deselects anything and I just never notice? Or does these modules in particular lack a “zero” state where the image isn’t affected so that even when out of scope areas are affected.

I chose the exaggerated example above because you could center the spot on black clothes set low scope and still see dramatic changes in the sky.

Only because Scope is not affected by this problem which is upstream in the pipeline.

The whole image is destroyed by this overexposure (in this case).

This does not generally prevent me from seeing what, in the process, can cause problems between LA and the recovery of highlights - but here this is paradoxically not the case here.

Jacques

That makes sense. But I’m also seeing something suggesting

Because I did some further experiments and it seems like

  • Color and light
  • Vibrance & Warm/Cool
  • Shadows highlights & tone eq
  • Dynamic range and exposure (DR compression slider)

All look good (no yellow) when enabled but as soon as any slider is moved in a positive or negative direction at even the smallest increment you immediately block out the sky in yellow.

I was mistaken that it was the color appearance and log modules only. It’s just that they act as soon as enabled whereas the others need adjusting. It only affects local edits though. The tools in the advanced tab (Color appearance and lighting) does not block out the sky.

So somehow all color tools in local adjustments start off with this data that blocks out blown highlights.

This is due to the fact that these 2 modules (Log Cam16) are “activated” upstream in the pipeline to calculate what I mention in “Scene conditions”

The system is complex, the pipeline is complex.

Perhaps it is possible to improve certain situations, not sure, but there the overexposure is monstrous. Hence lower white point and use Color Propagation

Thank you again :wink:

The raw file used is just an extreme example. Many files are affected but areas may be a metal highlight on a car or some other small thing in most files.

Retinex in advanced tab doesn’t produce yellow, Retinex in local adjustments does at strength 1.

Contrast by detail levels is interesting because it’s dependent on scope. Even when set to neutral changing the scope means increasing or decreasing yellow. All the other local adjustment tools produce yellow instantly when set to their weakest settings. The exception is the detail tools, like sharpness, blur-noise-grain.

All this suggests to me that there is something happening really early on for local adjustments that produce this yellow. Something that doesn’t affect the detail tools mentioned but all the rest.

Wild guess but perhaps white balance interacts in a strange way with highlight corrections and local adjustments. By setting a low wb temp 4000 or so or doing a adaptation in Advanced > Color appearance you can bring the yellow down in the sky and it looks impressively detailed, it’s not clipped worse than before or broken as such, just completely yellow. I’m not suggesting this as a work around but it shines some light at the problem.

I’m not expecting this to be solved any time soon but I just want to document as much as possible what’s going on now that I’m looking at it. Maybe someone will stumble on the solution.

1 Like

@nosle

I think, to check, to have found where it comes from…
This is a gamut limitation problem in the highlights.

With your image, go to “settings”:

  • Specific cases => Avoid Color Shift => XYZ Absolute

I think, it solves (in most cases ?).

Can you check ?

Thank tou

Jacques

I change, in branch “lacam16n” the default setting “Avoid color Shift” to XYZ absolute.

Jacques

yes the yellow disappears! It also has other complex mostly positive results. Just short comments will test more later.

  1. Yellow disappears
  2. Source data adjustments get new powers.
  3. Scope affects sky! At least with source data adjustmetns Log checked

Edit: A long time ago I had issues with local adjustments log. It created pink pixel artifacts. This was also solved by changing Avold colour shift.

@nosle

Thank you :slight_smile:

So, to check but I think that by default, this choice among the 5 is the “best”… It costs a few milliseconds…

Executables are in progress.

Jacques

2 Likes

@nosle @Lawrence37 @jonathanBieler
I did various tests and in all cases (the ones I tried), the result is better with “XYZ absolute” than “Munsell Only”. Of course XYZ Absolute also runs Munsell.
This concerns Cam16 and Log but also all other tools (except Denoise and Sharp)…
Of course to be confirmed by you or other users.

The increase in processing time depends on the size of the image (that of the RT-spot - if “full image”) and the number of RT-spots. It can be estimated between 100ms and 500ms depending on the case (for TIF/JPG). An average value of 200ms seems to me to be the current conditions of use. Which seems acceptable to me given the increase in quality.

I had planned these 5 cases: “none”, “Lab”, “XYZ absolute”, “XYZ relative”, “Munsell only”, from the beginning of LA in 2014/2015.

I suggest putting it by default in “Settings”, the user can obviously always change it.

Jacques