Question regarding dual demosaicing

I have noticed something odd with the new demosaicing methods RCD+VNG4 and AMAZE+VNG4. It’s not necessarily a problem, I am more curious as to why I observe the following phenomenon: When using one of the +VNG4 options I notice the colors become less saturated, and sometimes, of a different hue. Furthermore, if I push the threshold to 100% I would expect the result to be the same as just using VNG4, but this is not what happens.

Now, this may be by design I realize, but I am curious as to why , if so. Or is this a side effect of how the mixing is implemented? I suspect it relates to the camera having an atypical color filter array, but not sure why RCD+VNG4 at 100% doesn’t just look the same as VNG4. Again, it’s not a complaint, as it can be a useful feature when adjusting the threshold to taste. Just wondering why for the sake of knowing. Also enabling match greens doesn’t affect the outcome.

(The photograph itself is nothing special, just a random test shot I took one evening a little while before sunset)

Pictures are attached below.

RCD alone

RCD+VNG4, but with the threshold set to zero. Looks identical to RCD.

50% Mixture of RCD and VNG4

RCD+VNG4, with the threshold set to 100%. I would expect it to be the same as VNG4, since setting to zero makes it the same as RCD. However this is not what happens

VNG4

You could turn the mask on to see what is going on…

I don’t think its a simple blending as you might have expected

https://darktable-org.github.io/dtdocs/module-reference/processing-modules/demosaic/#dual-demosaic-algorithms

What did you mean by atypical CFA? What camera is this?

A few points i’d like to commen on:

  1. As mentioned it’s not “simple blending” so the 50% assumption does not hold in the way you expected. It’s a threshold to calculate the details level.
  2. Again- as mentions above - the mask will show you the regions

What about the hue shift you observed … i am very interested in the raw file to check this myself, it might be due to the special sensor as @kofa suspects but also i am aware of such shifts due to vng demosaicing at least locally.

What did you mean by atypical CFA ? What camera is this?

It’s a rather unusual and old camera, the Sony DSC-F828. It’s unusual because the CFA has two different greens, one the more typical green, and the other a blue-green (‘emerald’). Sony made one sensor to use this CFA, the ICX456 sensor, which was available on its own for scientific applications and in the DSC-F828 camera.

I bought it earlier this year since I got a cheap price, and was curious to play around with a camera using such a sensor. Also it has the ability to lift the infrared blocking filter from the sensor, which is cool since I do a lot of IR photography. The IR mode is crippled by limitations natively (it was intended for night vision) but a small magnet easily switches the filter in any mode. But I wouldn’t really say the camera is worth running out to buy one if you have a decent camera. The RGBE sensor didn’t have such a dramatic effect on color accuracy that a modern DSLR or mirrorless can’t perform equally or better and compatibility of its raw files with software is an issue due to the CFA.

I tried uploading the raw, but since it uses an .srf extension the forum won’t allow it since it’s not on the list of allowed formats. If there’s another means of uploading the file or sending it to you that you prefer, let me know.

Like I said, it’s not really a problem per se. Just curious as to how darktable treats VNG4 differently when it is used as the sole method vs as a secondary method. I suppose I should try digging into the code on github!

I believe this was ported from RT. I have thoughts but let’s ping @heckflosse for more informed ones. :stuck_out_tongue:

Yes, i was ported from rt and i did the work :slight_smile:

That’s why i am asking for the raw. (You could use any file sharing service or email hanno@schwalm-bremen.de) i would also like to test and verify rcd on that image …

1 Like

If I am not mistaken, *.zip files are accepted. Just place the raw in the container.

@bkv, i took my coffeebreak and looked for the sensor specs. Indeed - as all bayer demosaicers we use in dt (i think it’s the same for RT) don’t use a 4-color codepath the sensor is sub-optimally supported. A color-shift must be expected and we will probably have more chroma noise than “necessary”.

I would not suggest to use any dual demosaicing at all as there is probably more high-frequency content (which means detail) calculated than real.

You can use the green equalizing, it should help reducing chroma noise. vng might be better suited than rcd or amaze. lmmse is available in master branch if you compile yourself but i don’t expect wonders here.

There have been vague ideas about rcd getting better 4-color support but that is very low in priority.

Sorry about this … nevertheless i would like a raw file :slight_smile: (If you have one with a lot of subtle green tones?)

A workaround might be to declare all the emerald sensels as “dead”, so their values will be ignored. This can be done in dcraw, option “-P”. I don’t know about darktable.

This may be informative, but obviously not a great solution.

I emailed you a couple raw files, including one that’s almost entirely shades of green!

Anyway, in raw therapee pretty much all of the demosaicing options totally fail for this camera. I was surprised to hear that you ported them from RT, since darktable does a much better job than RT. RT’s options typically produce a mess of red and blue splotches at worst, and at best, an image with tons of maze like patterned noise. Your darktable implementation however , at worst, produces less saturated images and minor hue shifts with dual demosaicing.

I was surprised to hear that you ported them from RT …

A port is not a copy&paste :slight_smile: There are substancial code differences - most notably the way the blending and blending mask generation is done - but i am surprised about having them such an effect.

I don’t know RT code (just the demosaicing stuff) and don’t use it personally but would suspect a problem to be elsewhere. RT does some processing by default that modifies cfa data before the demosaicer stage.