Diagonal interpolation correction artifacts with AMaZE Demosaicing


#184

Good findings @agriggio and @LuisSanz. I had a suspicion that it had to do with wb and highlight mode. That is why I choose to compare output using that particular PlayRaw sample with its narrow spectrum, color imbalance, clipping and noisiness. I just didn’t understand what my own intuition was telling me or how to express it into words :blush:.


(Elle Stone) #185

You can’t use RT to set the white balance and then somehow squeeze out the RGBG multipliers. Most raw processors provide several ways to set the image white balance, including setting the RGB multipliers. RT does not. I don’t know why, but maybe this is tied up with how RT pre-white-balances or something.

Across the various raw processors, the “meaning” of RGB multipliers is more or less constant, meaning setting the same RGB multipliers in darktable, ufraw, dcraw, photoflow, photivo produces pretty much the same final image colors. But as RT doesn’t provide access, you can’t share the same multipliers when comparing output from RT and other raw processors. The best you can do is hope that the presets and “camera” are the same for the raw processors you want to compare.

Unfortunately, the temperature/tint slider values for the various raw processors don’t correlate at all. The “same” temp/tint taken from one raw processor and fed to another will produce more or less different colors.

This page has summary information for various raw processors’s options for setting white balance. PhotoFlow wasn’t released yet when I put together the information: https://ninedegreesbelow.com/photography/linux-raw-processor-review.html#white-balance

When using dcraw, I find it convenient to get the white balance multipliers from UFRaw and then copy them over to dcraw using the “-r” multipliers. Open UFRaw, set the white balance you want, then click on “Options” and select the Configuration tab - the white balance multipliers are available under “”.

The exif data in the jpeg comes from the raw file, and shows the white balance for various shooting conditions such as daylight, cloudy, etc, as given by the camera manufacturer. Use exiftool at the command line to examine the full contents of the raw file and the corresponding jpeg: “exiftool filename.ext”


(Alberto) #186

Hi @LuisSanz

I hope you don’t mind if I went ahead and tried to do this my own way :wink:
I just pushed my “hack” to the rcd-demosaic branch in RT. It basically tries to compensate for the difference in highlight scaling between dcraw and RT. I would greatly appreciate your opinion on this :slight_smile: (Also, please keep in mind that I only took some undergraduate courses on image processing a long time ago, and I’ve never worked with image manipulations since… so what I’m doing is dictated purely by “gut feelings”, and as such might be totally off!).

Anyway, for the curious, here are the results I get on the couple of test raws you posted above:

  • RCD in dcraw:

  • Original RCD in RT:

  • Tweaked RCD in RT:

  • RCD in dcraw:

DSC_0934-rcd-dcraw

  • Original RCD in RT:

DSC_0934-rcd-RT-orig

  • Tweaked RCD in RT:

DSC_0934-rcd-RT-tweaked

Best,
Alberto


(Carmelo Dr Raw) #187

I have just finished introducing the RCD demosaicing into photoflow. The implementation is based on the RT version, adapted to process small tiles in parallel.

Moreover, and following what @agriggio has reported, I have modified the RAW data clipping behaviour in photoflow, which results in much lower purple fringing around dark objects on clipped backgrounds. The improvement is actually visible for all demosaicing methods.

A small technical detail: I had to modify the code here and here to avoid inf values when the coefficients at the denominator are either equal but opposite, or very small.

Here is a comparison between Amaze and RCD with the new RAW clipping method:

Amaze:
DSC_0934-amaze-clip

RCD:
DSC_0934-rcd-clip

More details can be found in this other post on pixls.us.

For those interested, the photoflow code can be found here.


(Alberto) #188

So, inspired by what @Carmelo_DrRaw did for PhotoFlow, I generalized my “hack” above so that it works with any demosaicing method, and not just RCD. I’ve just created a new RT branch called highlight-smoothing (for a lack of a better name…suggestions welcome btw) that tries to get rid of the bad purple artifacts described above.
There’s a new option in the RAW tab, under “Preprocessing”, called – guess what? :slight_smile: – “Highlight Smoothing”.

Here are a couple of screenshots to show what it does:

  • AMAZE demosaicing, highlight smoothing turned off (the default, corresponding to the current behaviour of RT)

  • AMAZE demosaicing, highlight smoothing turned on

Now, if you are interested in this, you are very welcome to try it out, but please be aware that this is very experimental. It might actually be a regression for some images, I don’t know yet. I would especially like to hear about artifacts that are introduced by this trick, and under which conditions. In particular, one area that needs special attention is the interaction with highlight compression and the various highlight reconstruction methods.


(Sebastien Guyader) #189

Will you apply this for x-trans interpolation as well?


(Alberto) #190

Yes, this is completely unaware of the CFA pattern – it does the same for all sensor types. I have tested only with Bayer cameras though, so feedback for other sensor types is welcome!

BTW, I should also add that you need to apply this on shots with significant portions of the pixels clipped, otherwise you won’t see any difference.


(Sebastien Guyader) #191

Cool Alberto. I have some images from Venezia shot facing the sun, with strong clipped reflections on water. It will probably help in such cases.


#192

Latest compiled test version ( 4b6b806 ) for Windows (x64. Gtk3.22.26) can be downloaded from here:
https://filebin.net/cru16paem7pqbcs9

No installer included. Extract the folder “RawTherapee_rcd-demosaic3” to e.g. your Desktop and run “rawtherapee.exe” inside this folder.
Cache and settings are saved into “%localappdata%\RawTherapee-rcd-demosaic”. It leaves your existing installation untouched.


(Carmelo Dr Raw) #193

@LuisSanz @agriggio
I have spotted a couple of numerical instabilities in the RCD implementation, or at least in the current RT version, which might lead to inf and nan values.

  1. in the lines like
float V_Stat = epssq - 18.0f  *  cfa[indx] * cfa[indx - w1] - …

the epssq factor is mis-placed, and should be put either at the end of the expression or in a separate statement, something like

float V_Stat = 0.f - 18.0f  *  cfa[indx] * cfa[indx - w1] - …
V_Stat += epssq;

The reason is simple: if you add a very small number to a relatively large one (in this order), most likely the small number will not make any difference, because there are no bits available to represent it. Consider this minimal example:

#include <stdio.h>
int main()
{
  static const float eps = 1e-5, epssq = 1e-10;
  float sum1 = epssq + 0.5f - 0.5f;
  float sum2 = 0.5f - 0.5f + epssq;

  printf("sum1=%e  sum2=%e\n", sum1, sum2);
  return 0;
}

At least on my OSX system, this code generates the following output:

sum1=0.000000e+00  sum2=1.000000e-10

You can see that the first sum is identically zero…

  1. when computing the direction like this
VH_Dir[indx] = V_Stat / (V_Stat + H_Stat);

I encountered cases in which V_Stat = -H_Stat, and the above expression leads to a division by zero. In my code I mitigated the problem by writing

VH_Dir[indx] = V_Stat / (fabs(V_Stat) + fabs(H_Stat));

but I am not sure if this is correct, therefore I would really like to have the option of the experts…


(Alberto) #194

hi @Carmelo_DrRaw,
thanks for the freedback! I had fixed some of these already in the latest version of the RCD branch, but it is totally possible that I missed don’t cases… I will take another look :+1:


(Carmelo Dr Raw) #195

As far as I can see, you have introduced two assertions for the finiteness of the values… what I am suggesting removes the risk of having inf, which should be more robust…


(Alberto) #196

I also added parentheses to force a specific evaluation order… but I admit I did not check formally if that is enough


(Luis Sanz Rodríguez) #197

@agriggio, @Carmelo_DrRaw, that’s great. Thank you very much for your efforts and for spotting that error. Nice to see those changes in the way highlights are smoothed both in RawTherapee and in PhotoFlow.

I made some changes to RCD some days ago and I have just released a new version in the repository:

  • Fix to avoid directional discriminations out of the [0-1] range.
  • There was no need to compute PQ_Dir and LPF for every pixel, so both are no longer calculated for the green positions of the CFA.
  • There is an overall speed up of about 21%.
  • VH_Disc and PQ_Disc are no longer needed as a buffer, hence there is a 40% saving in memory consumption.
  • There are no changes in terms of image quality.

@Carmerlo_DrRaw, I didn’t know the position of epssq mattered. Anyway, a more exact option would be:

        V_Stat = MAX( - 18.f * cfa[indx] * cfa[indx - w1] - ...  + 1.f * cfa[indx + w4] * cfa[indx + w4], epssq );
        H_Stat = MAX( - 18.f * cfa[indx] * cfa[indx -  1] - ...  + 1.f * cfa[indx +  4] * cfa[indx +  4], epssq );

        VH_Dir[indx] = V_Stat / ( V_Stat + H_Stat );

The same should be done to P_Stat and Q_Stat. You can check it in the repository:


#198

Latest compiled test version ( 6512b9f ) for Windows (x64. Gtk3.22.26) can be downloaded from here:
https://filebin.net/k7spq7kqi3rsy838

No installer included. Extract the folder “RawTherapee_rcd-demosaic4” to e.g. your Desktop and run “rawtherapee.exe” inside this folder.

Cache and settings are saved into “%localappdata%\RawTherapee-rcd-demosaic”. It leaves your existing installation untouched.