Diagonal interpolation correction artifacts with AMaZE Demosaicing


(Samuel Chia) #1

Hi everyone, I am new here and was referred by Sandy McGuffog from the thread I posted over at the luminous-landscape forums. In that thread I describe in great detail my observations about how different demosaicing algorithms render detail and the issues I observe with each.

To summarise: Overall I find AMaZE to give me the best quality, but it is plagued with artifacts that appear like 45-degree diagonal ‘ringing’, and it was Sandy that turned me on to the term ‘diagonal interpolation correction’. I wonder if this is what is causing the artifacts that I see along diagonal high contrast edges when using AMaZE demosaicing.

To copy the relevant bits of my post from there to here: I have since noticed some disturbing artifacts with AMaZE demosaicing too - while it produces the best overall rendition of fine detail that has minimal mazing and zipper artifacts, some weird patterns form along diagonal edges of relatively high contrast. Note that you will not see these artifacts in edges that are vertical or horizontal or almost that. They are most obvious in diagonal edges (see first attached example, red arrows point to where one might see these artifacts). It is a sort of stair-stepped ringing artifact, with lines of varying contrast at exactly 45 degrees. I do not mean the different kind of digital jaggies, which is what happens with DCB demosaicing for example that is more coarse (you get 2x2 pixel blocks of steps rather than 1 pixel steps with AMaZE, which is the finest possible step). It’s a single pixel wide line of darker dots, always a perfect 45 degree slant, and these lines often appear next to each other and overlap, similar to the kind of ringing artifacts one might see with Lanczos 8 interpolation. I wonder if the anti-zipper smoothing algorithm of AMaZE is the cause of this kind of artifact, making zipper artifacts - alternating dark pixels at a 45 degree angle - smooth perhaps means joining them up. Another artifact that is observed is a dark single pixel tends to appear in the corners of edges, like on the tip of a leaf. They are peppered throughout the provided example.

When I capture sharpen with Focus Magic, it enhances the stair-stepping to a very high degree, making edges look to have strong ringing artifacts. The black-dots artifact is also enhanced to a large degree. Using the blend-if sliders to tame the dark halo reduces the effect, but does not eliminate it as it is present already. I’ve noticed that using Topaz Detail’s Deblur function, the ringing artifacts are hidden to a very large degree, except on some really problematic edges. I wonder if Focus Magic is creating these ringing artifacts - but a sanity check on a DCB demosaic version shows no such 45 degree diagonal lines. I also noticed that the thin-ness of edges, if there is indeed such a way of describing how crisp an edge is, is much finer than what I can achieve with even radius 1 in Focus Magic. The result is that edges appear to be more crisp, though at lower percentage viewing on monitors, the fatter halos with Focus Magic gives the sensation of a ‘sharper’ image, but when upsampled ~400% - 800% for printing larger and to serve 600ppi data to my Canon printer, Topaz Detail’s Deblur (or Infocus) produces a visible cleaner edge which lends itself to more (better) output sharpening.

I am providing here a few full resolution examples to show what I am referring to, including links to the original raw test image. These are layered tiffs (warning: huge) containing the full resolution renditions, converted with different demosaicing from ACR 9.1.1, Raw Therapee and Capture One V10, and then a copy of each with Focus Magic sharpening applied. The layer names should be self-explanatory. The bottom set of layers, without the ‘FM 1 300%’ suffix in the layer name, is just converted from raw, with the white balance and exposures matched as best as I can, with zero curves, sharpening, noise reduction and all other adjustments zeroed out. You can try your own editing on the sample raw file if you wish, if you don’t trust my layered tiff and wish to come to your own conclusions.

Raw file: https://drive.google.com/open?id=0B5AXKSbQEPuFZHBldXVSSi14eG8

Layered Tiff (1.3GB) This is the above raw image converted. I included the AMaZE rendition sharpened with Topaz Detail’s Deblur as a comparison to the Focus Magic Sharpened layer. Toggle the layer visibility for best visual comparison (easier to see than the side-by-side example crops below) Note that I used Focus Magic at 300% amount to enhance the artifacts of the sharpening process, and I NEVER use such a high amount ever for capture sharpening, but the artifacts are still there, just less visible/amplified. That is, the width of the halos, and the amount of ringing, is the same but with less contrast only at lower amount percentages. Pan about at 300% view or even more - there is a fascinating minefield of artifacts to look at. Flip between the unsharpened and no noise reduction originals (bottom layers) and the sharpened layers. It’s interesting to see how each demosaicing algorithm renders detail on a real world image, rather than on some synthetic target. Some leaves show significant mazing around them in the blue sky region, some leaves show significant zipper artifacts. Some don’t show any. Complex results.

https://drive.google.com/open?id=0B5AXKSbQEPuFU1hNMnhxNXQ5Ujg

Layered Tiff (2GB) This is a different example, a frame from a Sony A7R II, underexposed and with extremely low contrast and shot at ISO 100 which makes it a good candidate to look at fine grained noise when the data is stretched quite a lot with a curves adjustment layer. AMaZE renders the grain of the image quite harshly compared to ACR, with random extra light and dark pixels peppered about. ACR has perhaps the best noise grain pattern, which is not just softer but the intensity of the salt and pepper noise is leveled out to around the average deviation. It looks like ACR has some noise filtering going on in the demosaicing, despite the noise reduction settings zeroed out. This example has large flat gray regions of almost no detail to allow a study of the rendition of grain patterns by the various demosaicing algorithms, but also some contrasty detail in the bottom corner to look at artifacts on high contrast edges too. Not as good as above example for looking at edge artifacts however.

https://drive.google.com/open?id=0B5AXKSbQEPuFbGJjWVBVZXNIdUU

Here is a crop comparison from the above TIFF, at 300% magnification of an area of interest. The red arrows in the AMaZE crop point to the diagonal artifacts:

For variety, here is another crop example, AMaZE vs DCB, sharpened with Focus Magic (Radius 1, Amount 175%), that show the diagonal line artifacts very clearly.

I would be most grateful if anyone has an explanation for the diagonal line artifacts with AMaZE, and if there is any possible resolution for this, like perhaps applying a less strong diagonal interpolation. Thank you!

Regards,
Samuel


Firefox "find" doesn't seem to work for searching a thread
Is there a way to eliminate this conversion artifact?
#2

I’m no expert, but I dabble.

It’s not easy to get good diagonals, especially at 45 degrees. My guess is that AMaZE is making an error when figuring out what the green value should be at the red and blue locations in the bayer pattern.


(Jacek Gozdz) #3

I was analyzing AMAZE code years ago, but I recall that it interpolates diagonals (as well as near Nyquist resolution details) using different algorithm than rest of the image. It looks like the interpolation is fooled by high contrast edge and works like unsharp mask (a little bit). This could be corrected by clipping the interpolated value using neighbours. Still it would limit microcontrast.

By the way - DCB is doing that


(Ingo Weyrich) #4

I tried to reproduce the artifacts using rts on-board sharpening methods but fortunately I failed :slight_smile:


(Samuel Chia) #5

Thank you everyone for your comments.

Just for the record, if it was not already clear, the artifacts are visible even with zero sharpening.

So it is impossible that Raw Therapee’s own sharpening will not create these artifacts, because they are already there to start with. One just tries to avoid enhancing them too much. And here is a screenshot from Raw Therapee with RL deconvolution sharpening applied. The artifacts are quite visible indeed.

Perhaps the authors can look into these issues, if time permits? It is greatly appreciated. I heard from Bart van der Wolf that Emil Martinec once mentioned AMaZE is not a complete implementation in Raw Therapee because it will run a bit too slow, I wonder if that is true. Perhaps then could we have a ‘high quality’ option for AMaZE, with full implementation, for those of us not pressed for time and instant previews? Also, perhaps it will be nice to have a slider to control the diagonal interpolation strength, if that is at all the cause for the diagonal line ringing artifacts.

I suspect that turning down the diagonal interpolation strength will cause 2x2 block jaggies to form like in DCB, and increase the risk of zipper artifacts. It may have to be a trade-off, or perhaps the algorithm can be tuned to treat higher contrast edges differently from lower contrast edges.

cuniek - thanks for the insight into that behavior. I wouldn’t call it like unsharp masking, it is almost like Lanczos 8 ringing artifacts. But only for diagonal lines. The different interpolation at near Nyquist resolution details could explain the random peppering of dark pixels that go almost black in color when sharpened. Impulse Noise Reduction does not cure this.


(Ingo Weyrich) #6

Since Emil wrote AMaZE some time passed and I spend a lot of time to increase performance of AMaZE. Means there is some room for improvements. Why don’t you ask him about the incomplete implementation?

Edit: I miss the comparison to dcb in last screen shot!
Edit: Also I don’t know what you want to show with the 1st picture of 3rd row in 1st screen shot (the Topaz Detail’s). Why no comparison to other demosaicers for this one?


(Samuel Chia) #7

That’s great, Ingo. Your efforts at improving AMaZE has left me most pleased. If you read my thread over on the luminous-landscape, you will see that I find AMaZE to be far superior to even the highly regarded Capture One V10 demosaicing, apart from these diagonal line ringing artifacts.

I will write to Emil to ask him for clarification, thanks!

I’m glad you found the comparison to DCB in the above message. That is an important, clearer example to show the ringing diagonal artifacts.

Also I don't know what you want to show with the 1st picture of 3rd row in 1st screen shot (the Topaz Detail's). Why no comparison to other demosaicers for this one?

The third row just shows that Topaz Detail’s Deblur function almost perfectly avoids sharpening and enhancing the diagonal line ringing artifacts that AMaZE creates. It is irrelevant to see how it works on the other demosaiced examples because we are not trying to improve those. Deblur is not always perfect however, Infocus gives better control (almost too much freedom), and subtle changes to the settings give huge differents in the result. It is very difficult to wield effectively.

I’ll much rather not have any diagonal line ringing artifacts generated by AMaZE to begin with. :slight_smile: There is no perfect solution to demosaicing, and I am not looking for perfection. AMaZE is good enough, except for this artifact. If it can be resolved, I believe it will greatly benefit all.


(Ingo Weyrich) #8

Great that you will write Emil. Much appreciated :slight_smile:
But I still miss the comparison between amaze and dcb in your last screen shot.
Maybe you can post a link to the raw file used in last screen shot. Then I can check.


(Samuel Chia) #9

My apologies for not linking to the raw image file. Here it is:
https://drive.google.com/open?id=0B5AXKSbQEPuFREptekxyM0N2YTA

The earlier raw file can also be used to detect diagonal ringing artifacts, and also mazing and zipper artifacts. I find it more useful. This second raw file is not as useful, but for the sake of completeness, to accompany the crop examples, here you go :slight_smile:


(Ilias Giarimis) #10

@samuelchia

  • I cannot replicate your screenshots … can you upload the RT settings you used (pp3 file) and RT version/compilation flags (Preferences.About.Version) ?.
  • I think sharpening algorithms are not a good way to detect demosaic artefacts because the sharpening artefacts are mixed with demosaic artefacts. We have to use something simpler (like contrast increase) if we want to better visualize the effect

What I see is that Amaze (and all demosaicers at a different degree) amplifies (or maybe creates ??) fringing at the transition areas from clipped to non clipped. Not only for diagonals … this happens at all directions … it just happens that Amaze and DCB are very similar for horizontal/vertical but Amaze is more offensive at diagonals.
I think a problem exists at near (or just over) clipping with the values that are considered as reliable while they are not.
So I think that we need to reevaluate the pre_demosaic WB factors that are calculated and used at the demosaic stage. Most times we see this fringing is with files where the pre_demosaic autoWB fails or is a lot different that the actually used WB (


(Samuel Chia) #11

Thanks for the insights. However, I respectfully disagree. It has nothing to do with the transition into clipped regions. The first raw has no clipped portions between the branches and sky and yet the diagonal artifacts are still just as present.

If you like, I can find some more non-clipped examples.


(Samuel Chia) #12

Ok, I found a beautiful one where there is no clipping.

Here is a link to a layered TIFF crop, showing AMaZE vs DCB, with black arrows overlaid on a different layer, indicating where I see the artifacts.

https://drive.google.com/open?id=0B5AXKSbQEPuFS01XMkNnejBySzg

The settings can be easily replicated in any of V4 or V5 of Raw Therapee. All settings are zeroed, demosaic algorithm is AMaZE with 2 steps of false color suppression (default), then DCB with enhancement checked, also two steps of false color suppression. Zero sharpening. I placed screenshots of the settings in layers. I’ve deliberately added no other settings to the comparison, because I want to evaluate demosaic performance only.

In fact, the sharpened examples with Focus Magic are totally ideal to detect where the diagonal line artifacts appear. I’ve not seen one case yet where Focus Magic created diagonal line artifacts which was not already there in the unsharpened original. Then look at the unsharpened originals to see what the artifacts are like from pure demosaic only, with no additional enhancements.

Raw file here: https://drive.google.com/open?id=0B5AXKSbQEPuFdkU0d2hSVndUX3c


(Ingo Weyrich) #13

Default for false colour suppression is zero


(Samuel Chia) #14

Thanks for the correction. Ever since I installed V4.2.1148, the default was 2 steps. That’s why I thought so.


(Ingo Weyrich) #15

Maybe you confused it with dcb iterations (which defaults to 2)? I can’t remember that default for false colour suppression was 2 in past.


(Samuel Chia) #16

In my experience, the default demosaic algorithm is always AMaZE, and the false colour suppression is always two.

I just installed a fresh version 5.0-r1-gtk3, and indeed the above is true.


(Ilias Giarimis) #17

Samuel, you missunderstood my post (not your fault … because I was not clear )

I wrote that I could not see the artefacts on the first sample … maybe because of the bad laptop display I am on theese days … things will be better at the weekend when I will be at my home PC.

I could not replicate your results (probably because I didn’t use “false color suppression” )

I have no experience with Focus Majic etc … only with RT’s sharpeners … and I know that RT’s RL deconvolution gives artefacts especially whith “Dump” = 0 as you used it.

On the second sample I saw that the most obvious artefact was diagonal edges with wrong color (easy to see when “false color” = 0)
I remember Emil wrote in one of his last messages that the first he could try to improve in Amaze (if he was active) was better color interpolation.
I know that Amaze is very sensitive on pre_demosaic WB applied. On both your samples the pre_demosaic autoWB calculated is a bit far from normal … so this could be a reason :wink:

I hope Emil will respond to your (our) call :wink:

BTW … if you use the Pixelshift Branch, RT has the option to display at 1600% … this will make things clear with screenshots


(Ingo Weyrich) #18

There must be something wrong. It should be zero. I also looked at the code (there it defaults to zero) and also tried a fresh version 5.0-r1-gtk3. It’s always zero here. As it should be. Strange…
What happens when you press the reset button of false colour suppression adjuster?


(Samuel Chia) #19

I just tried hitting the reset button and indeed it defaults to zero! I see that the processing profile that is loaded is ‘custom’, not ‘neutral’, each time I open a raw file. I think that may be why - I must have changed that setting in the past.

So this new question springs forth - is a setting of ‘2’ too invasive for false color suppression? I (must have) used to think it offers the best balance of removing the most visible artifacts, while barely introducing smearing of fine detail. I notice colors tend to ‘bleed’ into fine, high contrast detail if false color suppression is too high.

Anyway, enough of that, back to the crux of the issue, the diagonal line artifacts. I’ve wrote to Prof Emil Martinec last evening. Hopefully he will be happy to shed some light on it too.


(Ingo Weyrich) #20

Great! I really was wondering why it defaulted to two at your system :slight_smile:
For a serious comparison of demosaicers false colour suppression should be zero to rule out artifacts introduced by values larger than zero (I can show you examples of this artifacts in case you are interested).

I hope, Emil will answer you. At least he answered on my Amaze optimization questions some years ago :slight_smile: