Issue with black blotches (help!)

stefan.chirila · February 9, 2021, 6:04pm

I got some weird black blotches when processing with
RawTherapee_dev_5.8-2735-g8bb06c2_20210121.AppImage

If I remove Capture Sharpening and Chromatic Aberration Auto Correct, the issue gues away. So perhaps it is broken in g8bb06c2_20210121

however the older RawTherapee_dev_5.8-2701-gb8d5a3d_20210102.AppImage
does not produce them.

RawTherapee_dev_5.8-2735-g8bb06c2_20210121.AppImage

RawTherapee_dev_5.8-2701-gb8d5a3d_20210102.AppImage

RAW: S7_09850.ARW (47.3 MB)
PP3: S7_09850.ARW.pp3 (13.5 KB)

Jade_NL · February 9, 2021, 6:11pm

The reason seems to be, or related to, the capture sharpening module.

I have a really strange motive all over the image when I open your image+pp3 in the appimage (stable, 5.8) and zoom to 100%. It disappears when I turn the capture sharpening off and then on again. It does not return if I now stop-start RawTherapee with this image.

When using the latest development (5.8-2780-g3b39ff384) I see the black blotches/squares (about the same as can be seen in your above image). Same thing happens when I turn on/off capture sharpening: Blotch is gone afterwards, does not return.

Here’s a screenshot of the motive I’m seeing in the stable, appimage version:

Interesting…

stefan.chirila · February 9, 2021, 6:15pm

thanks for checking it out! in my case I have to turn off both capture sharpening and aberration correction in order for them to go away. previous version works fine

heckflosse · February 9, 2021, 6:26pm

Thanks for reporting. I can reproduce the issue, but only when using Amaze+Bilinear

stefan.chirila · February 9, 2021, 6:31pm

@heckflosse in my case, using version g8bb06c2_20210121 I tested all the demosaic methods and they all gave the issue

heckflosse · February 9, 2021, 6:31pm

Disabling highlight reconstruction fixes it here.

stefan.chirila · February 9, 2021, 6:33pm

Can confirm disabling highlights reconstruction does fix it. Yet version 5.8-2735-g8bb06c2_20210121 had no issue even with highlights reconstruction.

heckflosse · February 9, 2021, 6:35pm

There was a change to highlight reconstruction method colour propagation not long ago
@jdc

stefan.chirila · February 9, 2021, 6:40pm

That would actually explain it, since you don’t even need to turn reconstruction off, you just have to switch it to Blend for the black blotches to go away.

Jade_NL · February 9, 2021, 6:47pm

I can confirm all @heckflosse’s comments and more:

Using the latest development version:

It only shows when using amaze+bilinear,
Turning off highlight reconstruction fixes it (only colour propagation shows this),
Turning off chromatic aberration-> auto-correction fixes it,
Turning off capture sharpening fixes it.

Also tried all the other modules that are turned on in @stefan.chirila’s pp3. None of those show any effect.

Thanatomanic · February 9, 2021, 8:33pm

I can reproduce this even more generally:

Start from Neutral
Choose any demosaicer you like
Push the White Point up to 5 or something high (this aggravates the effect)
Set Highlight reconstruction to Color propagation

Boom, artifacts.

Thanatomanic · February 9, 2021, 8:54pm

I have traced it to this commit by @jdc backporting something from ART, so @agriggio might also want to have a look for himself https://github.com/Beep6581/RawTherapee/commit/b554f52ab9136ad0333564b385f45d8196bd7b4d

What I don’t understand is why a core method was casually changed without mention anywhere.

heckflosse · February 9, 2021, 9:01pm

That I also don’t understand. Even more, by that change it got a serious slowdown…

jdc · February 10, 2021, 5:22am

Sorry, its my fault.
Probably the best solution is to disabled this improvment
Promised, i won’t do it again
Sorry again
Jacques

Thanatomanic · February 10, 2021, 6:14am

Well, let’s Alberto have a look first. If things are not problematic in ART, then we might as well revert. But if things can be fixed, there may be a very nice improvement to the reconstruction.

jdc · February 10, 2021, 6:23am

@Thanatomanic @heckflosse

This change followed a discussion on Darktable in january, and Alberto @agriggio made proposals to improve HR. They have been tested by ART users, and by myself. The improvement is significant.

But, it leads to a slowdown and in some cases to bad behaviour.

Considering the positive opinions, I didn’t go through the “issue” stage of RT, it’s a mistake? but who doesn’t make mistakes?

Sorry again

Jacques

agriggio · February 10, 2021, 6:24am

I tried reproducing in ART but I couldn’t, fwiw

nosle · February 10, 2021, 7:50am

I’ve also seen this. I use RCD/vng demosaic. It appears in the highlights with colour propagation.

Thanatomanic · February 10, 2021, 8:03am

@agriggio I can reproduce it in (edit: current dev build of) ART, can you try with this arp? S7_09850.ARW.arp (10.8 KB)

agriggio · February 10, 2021, 8:29am

Confirmed, thanks! Here’s a fix (the patch is for ART, might need to be adapted for RT):

diff --git a/rtengine/hilite_recon.cc b/rtengine/hilite_recon.cc
--- a/rtengine/hilite_recon.cc
+++ b/rtengine/hilite_recon.cc
@@ -951,7 +951,7 @@
 #endif
         for (int y = 0; y < H2; ++y) {
             for (int x = 0; x < W2; ++x) {
-                guide[y][x] = Color::igamma_srgb(Color::rgbLuminance(rbuf[y][x], gbuf[y][x], bbuf[y][x], imatrices.xyz_cam));
+                guide[y][x] = CLIP(Color::igamma_srgb(Color::rgbLuminance(rbuf[y][x], gbuf[y][x], bbuf[y][x], imatrices.xyz_cam)));
             }
         }
     }