About RCD and other demosaicing methods

Hi all. After being away for a while (I had to cope with three little kids and tons of work at the same time :sweat_smile:), I feel like coming back to play with image processing algorithms.

I have seen some ongoing discussion around RCD. May I ask what is the current situation in RawTherapee and if there is something I can do to help?

This Christmas I have been coding again. I tried out some ideas and came up with a new version of RCD demosaicing. It marginally improves almost every aspect of the interpolation but still needs a bit of testing. The underlying principles of the algorithm remain the same, so don’t expect big improvements.

I have been thinking too about how to improve AMaZE. In the end, the pipeline of AMaZE and RCD is pretty much the same and some parts are interchangeable. Particularly in AMaZE, I believe it will be pretty straightforward to change the green pixel estimation method from Hamilton-Adams to something else and to change the red and blue channels demosaicing for the one in RCD, which has noticeable less artifacts.

If the green pixel estimation is good enough, it will be possible to get rid of the diagonal corrections, which add computational cost and don’t seem to perform well in some cases. In the end, maybe it would just make sense to create a new algorithm since only the directional strength estimations would remain.

I have also seen that VNG-4 gets quite a lot of usage, mainly because treating a Bayer CFA as a 4-color CFA removes many high frequency artifacts. If there is any interest in a better 4-color demosaicing algo, I had one from many years ago which performed better in almost every aspect. I thought VNG-4 was there just to support very old cameras, not for actual use with current ones.

One thing I would like to ask is if the engine of RawTherapee is separable. I wouldn’t like to have to compile the whole project and open the interface to test every change I make but I also see that coding for the dcraw pipeline causes some issues later on.

If there are other areas apart from demosaicing where I can help, let me know.

10 Likes

Welcome back! See https://github.com/CarVac/librtprocess. (I changed the category and added tags.)

Might be less confusing on the naming and selection front.

2 Likes

@heckflosse should probably be the one to reply, but he did indeed split out the RT demosaic operations and a few other things to this library:

A few of us use it in our own projects; @CarVac’s Filmulator and my rawproc are the two I know about.

Edit: @afre, you’re too quick… :stuck_out_tongue:

1 Like

You’re an hour late. :stuck_out_tongue_closed_eyes:

1 Like

I don’t think @heckflosse ever switched RawTherapee itself onto using librtprocess, possibly because I never bothered to port the dual demosaic functionality over into librtprocess.

Thank you very much, @afre and @ggbutcher. I’ll play around with the library this weekend.

RCD got merged into darktable master in the past few days. I tried it on some high ISO images and the results were quite good.

1 Like

Hi @LuisSanz,

Thanks a lot for your work first of all! RCD is now my default demosaic method, possibly combined with bilinear when I want/need to pixel peep (which is not often, I admit). If you are willing to take suggestions/requests, here are a couple of things that come to my mind:

  • have RCD produce a smoother output in low frequency areas. You can do that already with the dual demosaic “trick” of @heckflosse, but having this native would be nice

  • improve high noise/high ISO handling. Currently I use lmmse for those, but it also comes with its problems (mostly some colour shifts in the darks which is sometimes annoying)

I hope these make sense – I only know the basics of demosaicing so I might be using the wrong concepts or terminology…

Also Siril

Hi, @agriggio !

Thanks for the appreciation and the insights. Could you provide some examples where a smoother output in flat regions is desired?

It can be smoother by using a 5x5 low pass filter (instead of the current 3x3) but that would come at the cost of some loss of resolution in other areas as well. The other approach I can think of (detecting flat areas and applying a different estimation there) is probably better done within the scope of postprocessing rather than demosaicing.

I think I prefer the idea of adapting it to work as a 4-color CFA algorithm, treating each green pixel in the 2x2 matrix independently, rather than changing the algorithm spirit. Its use case will also be wider as a fully featured alternative to VNG-4.

As for noise performance, it probably cannot be done without a complete redesign. RCD tries to follow edges and to smooth the output, and acutance in noisy areas needs exactly the opposite.

In the end, it’s difficult to have an all-in-one solution that works for all the use cases. Astrophotography needs are far from those of architectural or landscape photography and doing these changes just alters the balance by edging the results to to one side of the spectrum.

Another way of thinking about the implementation of the demosaicing, if you don’t mind an extra layer of user experience complexity, is to provide some parametrization adjustable via the interface instead of several different algorithms.

In the end, most methods (AMaZE, LMMSE, IGV, RCD, HPHD) work in similar fashion:

  1. Estimate the horizontal and vertical directional strenghts
  2. Estimate horizontal and vertical green pixel values
  3. Interpolate the green channel using the values from steps 1 and 2.
  4. Interpolate the red and blue channels.

These steps can be separate into functions to let the user do the choice via a dropdown. As an example:

Step 1

  • AMaZE - Works nicely with moiré
  • LMMSE / IGV - More suitable with high noise
  • RCD - Fewer artifacts, but ignores moiré.

Step 2:

  • Hamilton-Adams (AMaZE and HPHD) - Sharper but more prone to artifacts
  • High-order interpolation (LMMSE and IGV) - Deals nicely with very noisy images
  • RCD - Smoothest, maybe more suited for aggressive post-processing and printing
  • Bilinear
  • Some other methods out there

Step 3 is more or less the same for all the algorithms.

In step 4 there is not much gain and I think RCD works a bit better than the rest.

2 Likes

I would like to invite @hannoschwalm, who ported rcd to darktable, and @lock042 from Siril, which uses rcd from librtprocess, to this discussion.

1 Like

FWIW, my pet project rawproc has a command-line equivalent, img, that can be compiled and linked without all the GUI mess, if you need a test harness…

2 Likes

Thank you very much, @ggbutcher. I’ll have a look at your code. It looks like a perfect starting point.

I recently refactored the parsing and processing, requires touching a few places to put in a new demosaic algorithm. After you decide how you want to work, I can make a branch and add a new algo, speedy-quick for me to do rather than you having to figure out all the details…

1 Like

I just pushed some small speedups, Hanno and I worked out, to librtprocess rcd_speedup_new branch

3 Likes

@heckflosse, may I ask a silly question? I was able to compile librtprocess without problem and I was wondering if there is a command line wrapper or a simple gui I can use to load a raw and obtain a tiff. I tried rawproc’s img, but could not get it to work. I wouldn’t like to bother Glenn as he’s busy with the 1.0 launch.

Not that I know of

dcraw?

Depends on what you want said wrapper to do.

I’ll try with dcraw then. Thank you both!

Libraw, unprocessed_raw if you want a bayered tiff

1 Like