Quick question on RT Richardson–Lucy implementation

@afre Sure. I mean that every processing (linear or not-linear) before applying RL changes the input for RL

You’re right, I was definitely speaking inaccurately above. RT doesn’t do “gamma correction” per se; the tone curve is the gamma correction. Thanks for your help!

2 Likes

Nevertheless, applying RL at an earlier step in pipe seems better. Thanks for bringing that to my attention :+1:

3 Likes

Well, wait a second. I may have spoken a little too quickly. Following the original paper, I want to do deconvolution on the linear data captured by the sensor.

So, I think right after the demosaicing you have RGB data that’s basically linear. (Someone please correct me if I’m wrong.) If we were to deconvolve on one of the RGB channels now, then I agree everything would work out mathematically.

However, RT deconvolves on the L channel. According to some data I found, the L channel is not linear in the number of photons captured. So it doesn’t fit the Poisson noise model used in the Richardson paper.

In particular, reflectance percentages of (12.5, 25, 50, 100) correspond to L values of (42.4, 57.0, 75.8, 100). Definitely not linear. I’m not sure how to work around this, or whether it is worth the effort. I’ll think a little and let you know.

You can definitely test this if you know some C++ (for RT) or gmic scripting.

I believe, though am not totally sure, that the conversion is going to depend on the illuminant used. That’s a bigger worry to me (regarding implementing that could eventually be used in RT) than doing the math/coding.

I am guessing that the illuminant of choice would depend on the working colour space from which you are converting and your understanding of profile conversion. It can get confusing at times. :stuck_out_tongue: gmic has

rgb2lab:
    illuminant={ 0=D50 | 1=D65 } |
    (no arg)
  Convert color representation of selected images from RGB to Lab.
  Default value: 'illuminant=1'.

which a bunch of us requested. BTW, rgb is assumed to be linear sRGB.

Bart on the Luminous Landscape forums says the gamma for L is essentially 2.2. More info here.

@heckflosse Would it be possible to check whether accounting for the gamma transform on the L channel further improves the deconvolution? So convert to L, transform using gamma 0.455, deconvolve as usual, then transform using gamma = 2.2 again? I think this may help just as much, or even more, as putting it earlier in the pipeline does.

Here are some examples from the LL forum post I linked. The settings are not quite the same, but this is the best I could find without actually coding it myself. The algorithm used is from 2014 but I don’t think any subsequent improvements have addressed the gamma issue.

nonlinear

linear

I’m currently testing some changes. I will report back later…

1 Like

I asked on StackExchange and got a good suggestion here: https://dsp.stackexchange.com/questions/60043/what-is-the-correct-way-to-apply-richarson-lucy-deconvolution-to-luminance-data.

That’s funny because I just tried to use RL on Y channel :slight_smile:

Here’s a first screenshot:
Bottom right: default RT RL deconvolution
Top right: Same as before, but RL applied on Y instead of L
Bottom left: As Bottom right, but RL applied before tone curve
Top left: as Top right, but RL applied before tone curve

If there is interest, I can upload the raw and also the tiffs to a filebin

1 Like

Imho top left of my latest screenshot has much less halos than the other ones. Though it would be nice to get this verified by other people…

1 Like

Doing it blind I strongly preferred the top row to the bottom, but top left and top right seemed comparable.

Looking after your post, it’s clear the object behind the hose is rendered slightly better in the top left. Also there’s less haloing on the left edge of the door.

1 Like

Agreed!

Also agreed!

1 Like

Using Y instead of L for RL deconvolution is a simple change (about 20 lines of code) , though we need a gui element to switch between both methods to ensure backwards compatibility which means some more lines of code.

Applying RL immediately after demosaic is a bit more work, but I’m willing to put some effort into that as well.

1 Like

Little internet data left so I cannot see the samples. What I came here to say is that I have done some experiments on filtering using Y at various encodings a year or so ago and it seems to give better results than L* and norm[1]. I haven’t found time to test and update my own gmic filters but looks like you have confirmed that it works at least for RL.

PS [1] I typically use norms but perceptually Ys can perform better. It depends on your goals.

1 Like

One may ask now: why the hell did RT use RL deconvolution on Lab L at all? : Well, that’s because it was implemented this way in 2013 (maybe even before), noone changed it and noone thought about that.

Thanks to @nik who brought that to attention we may get at better RL sharpening in RT 5.8 :+1:

3 Likes

It’s 2019 folks.

I keep repeating myself, but convolutions (so, deconvolutions too) are operations that need to conserve the energy of the signal. As such, they need to operate in Hilbert spaces of square integrable functions, in order to respect the Parseval’s identity.

That theorem is 215 years old, it’s about time image processing begins to care about it. In image processing, such spaces of square integrable functions are scene-referred because the energy to preserve is the one of the light spectrum.

Lens blur is modeled mathematically by a convolution, and it does not blur gamma-encoded garbage but light emissions, aka photons characterized by their energy. Light is represented in digital imagery by scene-referred/scene-linear encoded RGB sensor readings. Deconvolving the lens blur in any space that got a non-linear transfer function squeezed in it is foolish, theoritically wrong, and visually unpleasant.

Dear opensource devs, please get your maths right before opening your IDE, because image processing is just about that, so you get no choice. That stuff is part of the first 3 years in the curriculum of any applied sciences university program. It’s not new. There are way too many engineers and scientists involved in opensource image soft to excuse any corner-cutting with theory. Apply the theory, then get proper results. Don’t, and expect problems. [2]

[1] Convolutions are a large family of filters using pixels and their neighbours, including:

  • blurring,
  • deblurring,
  • local averages,
  • guided & bilateral filters
  • highpass and lowpass filters,
  • wavelets/Fourier spectrum decompositions,
  • rotations,
  • interpolations,
  • conform distorsions,
  • etc.

[2] And when you do get problems, don’t add thresholds and opacities tricks in your broken code to hide halos, fringes and funny stuff under the carpet. Go fix your models instead, or you will loose everybody’s time in the community.

4 Likes

Hi Aurélien. While I largely agree, I’ve found references suggesting that certain wavelet transforms are better done in a perceptually uniform space such as LAB, which is not linear.

For example:

JPEG and other lossy image compression algorithms depend on discarding information that won’t be perceived . It is vital that the data presented to a JPEG compressor be coded in a perceptually-uniform manner, so that the information discarded has minimal perceptual impact. Also, although standardized as an image compression algorithm, JPEG is so popular that it is now effectively an image interchange standard. Standardization of the transfer function is necessary in order for JPEG to meet its users’ expectations.

Do you agree? The argument may extend to e.g. guided filters for edge detection; I’m not sure.

Mostly has to do with backwards compatibility. I tend to go the gmic route, which provides a high degree of freedom, but I have to hand do everything, so I am mostly lax in getting it right all the way through, at least for the PlayRaws.

It depends on what you want to achieve. See: New shadows/highlights tool with enhanced halo control. It is only a part of what @Carmelo_DrRaw has been working on. He has a way to preserve the ratios at the end, though one might disagree with the method.