Diagonal interpolation correction artifacts with AMaZE Demosaicing

One of the good test files is the one from the Canon 5DsR on DPReview of their typical studio image. The central tiny B&W text and also the etching drawing are very good stress tests for demosaicing. Not to mention the issues with noise in the shadows at base ISO.

https://www.dpreview.com/reviews/image-comparison/download-image?s3Key=9ef169a9dc1e4a639994abc63e7ff240.cr2

No, I’m not, but I haven’t looked for them lately. I read a few papers some time ago, but I haven’t seen any open source implementation yet. Any ideas?

RCD creates that kind of pattern with noise too.

VNG4 treats the two green pixels in the 2x2 Bayer CFA as separate channels. This way it avoids many artifacts, but there is an important loss of resolution.

If you are interested, I can easily fine tune RCD for high ISO images so that it keeps the noisy pixels isolated instead of creating lines between them, but I would like to keep it as a separate algorithm.

That sounds perfect! Two approaches, one for max resolution and one for making noise look pretty, whist sacrificing resolution.

Let me know what do you expect. Any info you can provide will be welcome, since I don’t do astrophotography myself. It would be great if you can show me samples of flaws you want to avoid, share some raws for testing purposes, etcetera.

Didn’t @Iain do something with moire reduction not long ago?

Edit: found the thread:

Yes, I tried IID briefly in one of the PlayRaws and initiated a light conversation on it.

PS That hatching pattern that @samuelchia showed has always annoyed the heck out of me. It affects me more because my aging camera produces noisy low resolution output.

FYI, I’ve done a bare-bones (i.e. unoptimized) integration of the RCD demosaicing by @LuisSanz in RawTherapee. If you are interested, you can find it in the rcd-demosaic branch.

4 Likes

@LuisSanz, to add further detail and examples regarding how various demosaicing algorithms render grain structure. I’m attaching annotated crops magnified to 300% for easy quick viewing, and also the raw file. It is from a Sony A7R II at ISO 800. It was shot on a Star Adventurer tracker which I aimed blind at Polaris because I was standing next to a cliff so the tracking is not perfect. I think this astro image is a reasonably good test because there is a fair bit of noise, smooth areas where it’s easy to see any patterning artifacts, and fine star details which tend to produce a lot of false colour artifacts.

In terms of noise pattern, I like VNG4 and Adobe Camera Raw/Lightroom’s result the best. Note the bright star in the upper right. VNG4 produces a zipper artifact along that edge while Adobe does not, bearing in mind that the Adobe algorithm is prone to severe zipper artifacts as demonstrated in the earliest examples. In this case it did not trigger with Adobe but in VNG4 we see it. I find VNG4 to be very prone to this kind of zipper artifacts too.

The Raw Pedia page recommends LMMSE or IGV for noisy images, which I find to be a poor recommendation. They both enhance the noise contrast more than VNG4 does, and perhaps the color artifact suppression is a bit strong and robs away some star colors. There are also random 45 degree diagonal artifacts usually of three or four pixels in length where the pixels are too dark in smooth (!) areas, which is something I’ve not seen before. Sometimes it is two adjacent pixels, or two adjacent pixels separated by a lighter middle pixel.

AMaZE has slightly less of the mazing pattern in the noise, DCB has a little more and AHD has the most. These patterns are impossible to remove with typical noise reduction tools in the raw converter or third party apps like Topaz Denoise. @afre, I feel your pain. AMaZE tends to pick up and increase the contrast of a random pixel here and there a bit too much making it too dark, while DCB’s micro contrast is smoother (i.e. less standard deviation of brightness of pixels in the dark star-less sky areas which is not featured in this crop illustration).

VNG4 produces a lot of green fringing false color halos around the stars. Adobe color fringing errors tend towards red/magenta. LMMSE produces even less false color, and IGV has the lease false color and color noise is strongly suppressed, but star colors seem to be destroyed also. DCB seems to produce the better balance of false color/color fringing artifacts, followed closely by AMaZE, but one can find stars where either one produces a better result.

Overall it’s close between Adobe and VNG4 when it comes to dealing with noisy data. The Adobe result has less zipper artifacts in the high contrast stars, eliminates hot pixels slightly better and generally has less objectionable color fringing artifacts. The VNG4’s green halos on high contrast edges (stars) I find to be objectionable, which is not removed by the false color artifacts suppression slider, but can be dealt with using the Defringe tool. VNG4 produces better star shapes in some of the smaller, fainter stars, which I appreciate.

If RCD is able to avoid the color artifacts better, and incorporate VNG4’s method of treating the green channel pixels as separate etc. etc, I think we may have a winner.

VNG4 does rob quite a bit of resolution away, so my current favourite technique is to render the image twice, once with DCB and once again in VNG4, and mask out the stars which will come from the DCB image, and the smooth low contrast areas will be from VNG4. I agree that a modification to RCD to handle noise patterns better should be kept separate, the loss of resolution and detail is pretty significant. I’d much rather have the final control of manually blending the two together for best results.


_DSC4132.ARW (81.6 MB)

2 Likes

A compiled test version for Windows (x64. Gtk3.22.26) can be downloaded from here:
https://filebin.net/x6uf72a6r1on1roj

No installer included. Extract the folder “RawTherapee_rcd-demosaic” to e.g. your Desktop and run “rawtherapee.exe” inside this folder.
Cache and settings are saved into “localappdata\RawTherapee-rcd-demosaic”. It leaves your existing installation untouched.

2 Likes

Big thanks to @agriggio for adding the code @TooWaBoo for the complied test version!

So I was able to repeat the test on the durain tree raw image. This illustration shows AMaZE, DCB (current best overall choice), RCD and VNG4. I did LMMSE and IGV but both were pretty awful compared to VNG4, removing color in fine details and resulted more noise than VNG4.

The first row is a straight raw conversion with a camera profile with a single point brightening tone curve with input 50% and output 75%. All other settings zeroed, no sharpening, no noise reduction, no false color suppression. As the differences appear very subtle, the second row has unsharp masking of 500% and radius 0.3 applied to exaggerate the differences to make evaluations easier on the eye. These are 100% magnification crops, upsampled to 300% with Nearest Neighbour interpolation. Please note that I do not apply this kind of sharpening for regular, normal editing.

The first most interesting observation is that RCD has the smoothest rendition of noise, even smoother than VNG4, which until now had the best noise rendition, and only in certain areas does the hatching pattern become visible. I was not expecting this! If a modification could be made to remove the hatching pattern, this could be the best result ever for noisy images.

RCD also has the least false color issues overall, by a very significant margin compared to AMaZE and DCB. Outstanding.

The diagonal zipper artifacts is greatly suppressed in RCD compared to AMaZE, but they are still present, and I see ringing artifacts still present. RCD also suffers from more mazing artifacts than AMaZE (see some of the leaves in the lower region). However, any artifact usually has far lower contrast than AMaZE so more real-world sharpening routines will probably not cause them to become too obvious, or probably not even at all.

DCB has no ringing artifacts and no mazing artifacts, only the occasional too-dark random pixel. Other than the false colors, which can be mostly avoided with two steps of false color suppression, I still think DCB overall provides a more artifact-free, natural looking result, albeit with slightly less ultra fine detail which loss is not visible at 100% magification, and certainly not even in the best prints possible today.

I really like RCD. If the diagonal zipper and ringing artifacts could be brought down to DCB levels, it could be the best demosaicing I’ve ever seen.

1 Like

Hi! Thanks @agriggio for merging the code into a new branch and @samuelchia for the testing. I believe there is a small bug somewhere in the implementation, since the code in RT creates zipper artifacts in the diagonals as Samuel noted.

Note in this sample that in the ouptut I’m getting from dcraw the straight lines and the edges are rendered smoother than in RawTherapee. I’ll have a look to the code to see if I can find the cause for the divergence.

dsc_7738_rcd-dcraw_res

At first sight, the implementation seems pretty straightforward, with my code almost untouched. Might it be because of the OpenMP declarations?

Hi @LuisSanz, thanks for checking!

Yes, I tried to keep it as close to the original as possible, intentionally. I think in general it is important to have the “pristine” version working first, before doing any optimization. I only added some openmp pragmas to parallelize stuff…

I just recompiled with openmp disabled in rcd, and I don’t see any difference… but I’m not an expert in this, so here are the two results for you to examine (I used the neutral profile)

Thanks @agriggio, I see tiny differences, which is easier to see in an exaggerated difference map of the two images overlayed - it looks like the differences caused by JPEG compression or maybe something else. Not the kind that would cause the zipper artifact bug. They also appear in the smooth areas of sky mostly rather than in the more textured parts.

A long time since I’ve done any programming but some of the later loops appear to read/write from/to the same array (which might be a problem in parallel, I really don’t know). Maybe it’s just precision differences…

1 Like

you’re totally right, of course :flushed:
I could say it was late in the night when I added the omp stuff, but truth is, I simply overlooked this… :wink:
I just committed a new version without parallelization of step 4. We can always optimize later.
Thanks again @garagecoder for spotting this!

2 Likes

That’s true. However step 4 has no influence on the green output, which is also slightly different. I believe the problem might be with V_stat and H_stat showing negative values or not being stable enough. I will try to compile myself RawTherapee to be able to do some tests and be sure.

well I don’t know the code enough (well… not at all in fact) to determine if the parallelization is really problematic or not, but I turned it off just to stay on the safe side.

I was unable to find the cause for the green channel differences between my implementation in dcraw and the one in RawTherapee. I ported the algorithm myself and the results were identical to those I posted before using @agriggio’s branch.

Is there any change in the pipeline the raw data goes through before the demosaicing between dcraw and RT with the neutral profile applied?

In the meanwhile, @samuelchia, I can send you a compiled dcraw. Which OS do you use?