Diagonal interpolation correction artifacts with AMaZE Demosaicing

Software RawTherapee
61

Thanks for the encouragement Ingo. :slight_smile: Although they clearly say there is no problems with AMaZE, only with Focus Magic. Indeed the problem often is being able to see the problem in the first place. That’s the bane of a scientist’s or engineer’s or designer’s life. Or pretty much everything else! If I could connect my brain to another person’s with electrodes or something that can send brain waves across and let them experience what I do, this whole exercise will be so much easier. Perhaps one day technology will take us there.

It is true that these issues are generally too small to bother with for most folks. Unfortunately my standards of work are pretty stratospheric. :wink:

I really appreciate it. Thank you for looking into this. I am also (finally) looking at the AMaZE and DCB demosaicing math in the code (from the link you provided above), and if I notice anything, I’ll sound it out too. But otherwise I would prefer to observe from this point, not being anywhere near an expert at demosaicing math or coding. Lest I begin to spout nonsense


62

I think I have you beat: trying being a writer when the new Dan Brown book or 50 Shades of Gray come out. They’re beat sellers, many many copies sold.

Yet I barely consider 50 Shades to be written in the English language. Madness!

63

Sorry, I don’t understand what you are trying to say here. Though I agree that most writing is worthless. Like most photographs (yes, even mine) are worthless.

64

I’ve done some demosaicing experiments in GMIC.

I found that diagonals benefit from interpolating the colour channels (or the colour differences) diagonally first, based on the orientation information from the green channel, and then horizontally or vertically based on green channel or local direction estimates.

Here you can see the results of two of my demosaicing experiments compared to other algorithms. My first experiment (Mine 1)has some ringing artefacts similar to those described in this thread. My second experiment (Mine 2a) renders less fine detail but has less ringing. It also has a refinement step (Mine 2b) which pulls out more details but will introduce zipper artefacts if overdone.Both experiments use the diagonal first interpolation I mentioned above.

1572×1016 561 KB

Raw file here: https://dl.dropboxusercontent.com/u/10782279/DSC_6553.NEF

1 Like
65

Fascinating!! Thank you for this example. Unfortunately for 2a and 2b, there is a lot of false detail and aliasing artifacts near and at Nyquist.

Also in both 2a and 2b, along the main diagonal lines, the line goes from sharp pixels to a fuzzier edge, then becomes sharp again, then becomes fuzzy again. ‘Mine 1’ does not suffer from that at all and actually, to me looks the best here. Even better than AMaZE which also has the sharp-fuzzy-sharp issue. ‘Mine 1’ looks to have much less severe ringing than AMaZE. Hardly anything really. That’s a stunning result!

Is this your own original demosaicing algorithm or did you modify/base it off an existing one?

Edit: The DCB example is without enchancement I presume? It looks awful. With enhancement it looks closer to ACR, which has the 2x2 pixel jaggies.

Edit 2: It looks like you modified AMaZE? If ‘Mine 1’ is AMaZE, but with diagonal line interpolation before horizontal and vertical, it looks to me to be an unqualified success! I think it’s just what I was looking for.

1572×1016 611 KB

66

It is a simpler algorithm and only looks to avoid zipper artefacts when choosing the interpolation direction, not false colours and aliasing. It is possible to add a consideration of false colours and aliasing when choosing the interploation direction.

In 2a and 2b the missing green values are interploated without considering the values of the red and blue pixels. In Mine 1 the algorithm considers the correlation between the colours which allows better details.

These are my own algorithms. Although I expect I have repeated things from other algorithms without knowing it.

Yes, that is correct. The DCB example has no enhancement.

67

Do you mind sharing a crop of this region using the ‘Mine 1’ algorithm? It is too difficult to judge the ringing artifacts on single pixel width detail, one needs more region of flat color on either side of an edge to see the ringing develop. As we see in the below crop, AMaZE produces the ringing as expected.

Thank you for the insight. Did you notice how AMaZE goes from magenta to green to magenta along those diagonals? While ‘Mine 1’ is virtually completely free of that color shifting issue, and is almost completely uniform and color stable.

Thanks!

68

Here is the crop. I used RT to convert the image without any false colour suppression and without any colour profile so that it is a fair comparison.

A diagonal line that is close to 45 degrees will go through mostly green pixels and then mostly red and blue pixels, so it is hard to get the correct colour. My demosaicing suffer from is to some degree.

69

Thank you. I may have to retract what I said earlier about the lack of ringing. I see that the ringing is actually worse in some ways with your algorithm, spreading further from the edge with higher contrast than AMaZE. Also, AMaZE produces better rendition of noise patterns and single pixel fine detail. In other ways your algorithm is better, see below.

True. Even so, your result is way cleaner than AMaZE regarding false colour in the diagonals. Very impressive.

70

Hello.

@RawConvert
I wish there was no artifacts, but they are there. Some of them are HUGE, like this moire and wrong interpolation directions (all images are AMAZE, between 200 and 400% zoom, with typical USM, no CA correction)

stairstep effect (where long nearly diagonal line is made of long, perfectly diagonal sections).

zipper artifacts (edge is not a line, but a
 zipper)

or 2x2 blocks (this is DCB)

Of course things are greatly improved when image is properly postprocessed.

@samuelchia
I think You can stop posting more examples, everybody here knows what You are reffering to, the reason for some argument is that examples You show are exhibiting visible artifacts only when oversharpened. In all other cases they are well hidden and even skillfull RT users have hard time spotting them. There is a bit of confusion when You write You dont sharpen that much in real life, and say You can not live with those artifacts (visible only when oversharpend). Showing an 8 pixel long diagonal edge that consist of diagonal lines and calling it a “problem” is not helping too.

To sum things up:

  • AMAZE is creating not perfect rendering of diagonal lines (period).
  • Me and @heckflosse will try to improve (but not at expens of other issues)
  • Problems You have can be minimized by proper image processing (as I showed before)
  • If You need help processing files, You should explain what is Your target look (seems You want to greatly improve local contrast, and this leads me to the question if you tried wavelets?) and target size. Most sigle pixel issues are not visible even when printed at 160dpi (I know this can start a new discussion, thats why I wrote most).
  • For fine art prints it is always necessary to do some manual retouching. Your needs seem to be extreme, be prepared to correct things using other software. Once again, it would be nice if You share Your experience with huge prints. How big they are, what was original resolution, what is actually visible on a print
  • Last but not least, the “ringing” you are reffering to is a result of natural, optical behavior of a lens. It is then obscured by bayer pattern on a sensor, amplified by demosaicing algorithm (which creates 2/3 of informaion You see on a photo - accept some imperfections), and then by sharpening.

I know this is not solving Your problem, but unless You have an idea how to improve RT code, all You can do (for now) is to hide artifacts and accept imperfections. Believe me - noone will notice if photos are great :wink:

71

“Sharpness is a bourgeois concept” (HCB)

I’m sorry, I couldn’t resist


1 Like
72

728×488 85.3 KB

Thurles, Ireland, 1952 © Henri Cartier-Bresson/Magnum Photos

Or as I like to think, “Guests at a Samuel Chia Private Viewing looking for Jaggies”

Sorry Samuel, I had to work out who HCB was, then I saw this photo, I couldn’t resist, no offence intended. Blame the previous post!

3 Likes
73

Don’t be silly :checkered_flag:

1324×768 44.6 KB

 
Sorry I’m a sheep, go with the herd :stuck_out_tongue:

74

On a more serious note, the last job I had was around a set of (18) images, very small, jpegs from a 1/2.3 inch point and shoot and the client :koala: wanted large prints for an exhibition. Yeah raw, sure. Anyway I personally don’t print them, but know the guy from the printing shop and the fractal tricks he uses. So off I went to create an automated action that could apply to the set as starting point. As you most certainly know the key word is dithering, enlarge, denoise, dither, mix, rinse, spit, curse, repeat, and so on trying to perfect the proccess. Grains’ my god

In this particular case most of the editing was around colour, as you can imagine there were a lot of oversaturated waxy hues, seas of beautifull noise and martian’s skin. Dynamic range, ha ha ha, a reign of white skies and endless dark shadows, beauty.

Recovering colour and figuring out a “palette” for the whole set was where I put the most effort. Perfection does not exist this was my main mantra to preserve sanity. Ended up remixing a homebrewed LUT and can honestly say I was quite happy with the overall result, on full disclosure one image was off (below)
 got to fix that 2 days after delivery; not that they even notice, but I knew.

1200×896 118 KB

And that was it
 Some weeks later I went to the gallery, with absolutelly no idea what I was going to encounter. Must say I was positively surprised as the printing guy managed to keep the thing together, specially regarding colour. Only if you went caresing the frame’s glass with your eyelashes you could see some minuscule “dirt”; took some photos for my own documentation and spend most of the time admiring the grain’s pattern from film prints on some 80’s stills nearby.

Like they say my 2 cents

1200×800 118 KB

1 Like
75

@cuniek @heckflosse
Thank you for your efforts at trying to improve the demosaicing. That is greatly appreciated.

It is absolutely true that such problems that I pointed out can be mitigated in a variety of different ways. In fact I was the first to point that out in my OP. But for the very same reason that I abandoned Lightroom’s demosaicing in favour of AMaZE, mitigating such issues gets to become a tiresome exercise. Minimising issues at the starting point makes digital processing life significantly easier, as per @chroma_ghost 's story.

It is also true that I have a critical eye worst than most. When I started photography I had very little money to spend on high-end equipment. I have a number of files in the 10-21 MP range, which are important photographs to me. While I know I cannot make 40x50 inch prints from some of them, some can be stretched to 32x40 inches. Some of my clients come to me with very low resolution images, and want to make large prints. Once I had to make two 1x1 meter prints from a 6 MP image. It was a real battle. The vector-based upsampling that Photozoom Pro uses was a huge help to maintaining the thinness of edges.

The point of all this is to avoid anything that shouts ‘digital’ in our photographs. Halos, mazing, zippers, pixellation, jaggies are all quite hideous in a print. Minimizing artifacts as far as possible makes them look very natural, even on close inspection, whist having low to invisible noise and with thin, crisp edges.

One can just avoid sharpening altogether and end up with a softer image but with no visible artifacts in a print. This is what I’m doing today to avoid enhancing the issues. But with a cleaner file, one can have a crisper looking print at the end of the day. While that may be insignificant to most, I am not interested in resting on my laurels. If something can be improved, why not?

I admit this is totally new to me. Could you explain or direct me to an explanation of this lens optical behaviour that creates ringing?

All in good spirit! :grin::joy: Sharpness is a bourgeois concept. We are at aristocrats’ level looking for things smaller than sharpness


Otherwise, tell me, why would the RT authors be so interested in functionality for at looking at 1600% for pixelshift? Why do we even need pixelshift? Look at those billboard size Shot-on-iPhone prints. Are those good enough? Maybe for some!

None taken :wink: Though if I ever had such a club, members will qualify only if they can see the jaggies with their own eyes, without further visual aid. :joy: LOL Regular corrective vision spectacles are fine. No bionic enhancements, binoculars, microscopes, loupes etc. *Joking!

76

Because it speeds up the evaluation process! It’s much faster to increase zoom factor than to move closer to screen!
Using mouse wheel is much easier than moving whole body. We’re just lazy :wink:

77

Of course that is the reason! :slight_smile: But what I meant was that if sharpness is not really important, why even do that? :wink:

78

Of course getting as much detail as possible out of the shots is the goal. Pentax would not have used pixelshift sensors if that would not be the case.

79

Thank you, that is exactly how I feel too.

80

Theory: Lenses have global and local contrast (that is why MTF functions are measured for different number of lines/mm). High local contrast is what we call “sharpness” (how visible small details are). Global contrast tells how big an area must be to becom really black (as black as a lens can render). This explains why thin, but not thinner than 2 pixels black branches on a white sky are not black (while there is enough data on bayer pattern), while 4-6 pixel wide are (the same wood, the same light, the same color, 
).

The bigger the contrast, the stronger the effect (I heard a term “light leaking”, not to confuse with “pixel bloom”).

This becomes more and more important to me as I started to compare Sigma 50mm ART (high local and global contrast lens) with 1938 Leitz Summar 50mm (low global contrast, moderate local contrast lens).

Lenses: Local contrast is acheaved by minimising aberrations and creating sharp, precise design. Global one, by having clean and smooth lenses with good antireflective coating, and no internal reflections.

Digital photography: Edges of big, dark areas on big bright backgrounds are always a little bit brighter, than they should be (USM fixes that, when properly used). AA filter on a sensor and demosaicing which mixes data from both sides of an edge, CA and maybe something else, makes it worse. Thats why the demosaicing alone is not to blame. It is sometimes confused by unexpected phenomena. The results would be better if lenses and sensors were perfect.