Diagonal interpolation correction artifacts with AMaZE Demosaicing


(Ingo Weyrich) #41

As a side note: The necessity to implement 1600% magnification in pixelshift branch proves that we are on a good way with pixelshift processing. Pixelshift is extreme pixel peeping :wink:


#42

It’s not extreme pixel peeping until you can fill the whole screen with one pixel.

(I didn’t zoom all the way in to a single pixel because that’s less interesting to see)


(Ingo Weyrich) #43

Ok, you beat me :slight_smile:


(Samuel Chia) #44

That’s great!

I’ve quite often been accused of doing too much pixel peeping but this is extreme indeed - and quite useless! LOL


(Jacek Gozdz) #45

Hello.

This discussion was quite interesting to me, as I considered myself The Great Pixel Peeper (first of this name). I don’t consider myself an expert, but I know “this and that” about demosaicing, as I am the author of DCB (both, the method itself and referencee DCRAW implementation, but not RT implementation). Keepeng that in mind:

  1. Stairsteps artifacts of AMAZE may seem a big problem if the image is developed to emphasize them, and extreme sharpening artifacts are overlapping with demosaicing artifacts (they are there, that is true, noone is denying that).

  2. One can develop RAW in RT to hide this kind of problems. This reqires knowledge and skills. I attach my approach, as a proof that sharpened, AMAZEd image zoomed 400% can still look nearly artifact free.

_MG_1570.CR2.pp3 (9.9 KB)

In other words, there is nothing in presented pictures that shows any real problem with AMAZE (and I say that as someone whose algorithm is considered direct AMAZE competition). Remember You will get some strange artifacts everywhere if You sharpen strong enough (micronoise, highlight recovery artifacts, sharpening artifacts, differneces in CFA between pixels …)

NOW!

If we look at this discussion as purely academic pursuit of excellence:

  1. There is always DCB :grin:

  2. Correct CA before demosaicing as ALL demosaicing algos are prone to errors when R/G/B channels are not properly aligned. Automated CA correction solves some problems but demosaicing author should always be aware of this phenomenon.

  3. Right WB applied before demosaicing is also critical.

  4. As I mentioned, it is possible to try to apply DCB’s remove overshooted pixels code. I wonder if it would fix the prolem. Anyone care to try?

  5. If you really want to see image that shows demosaicing artifacts - feel free to use this one for testing. There is severe moire, wrong interpolation directions, stairsteps…

    https://filebin.net/o1lorga3utd3co41

I hope I helped.

Jacek Gozdz <afjg.pl>


(Samuel Chia) #46

Thanks for sharing Jacek. I found insightful to read your comments, especially since you are the inventor of the DCB method of demosaicing.

Excellent job. Though I can still see the diagonal ringing line artifacts in your edited crop, especially in that most problematic zone on the thicker branch. It is not objectionable as is, but when I use what Bart van der Wolf considers the best tool for capture sharpening, Focus Magic, it is greatly enhanced, which annoyed me to no end. Even without using Focus Magic, using Photozoom Pro to upsize for high quality fine printing, also causes the artifacts to become a little more visible.

That’s very noble of you to say :slight_smile: But I wouldn’t have taken so much time to document and explain this issue if it didn’t actually present a real problem in my prints.

True indeed, except that it does have a little more mazing and zipper artifacts, and also diagonals tend to block up into roughly 2x2 pixel blocks, looking a little like fine jaggies. But usually that is less objectionable than AMaZE’s ringing artifacts.


(Jacek Gozdz) #47

… but when I use what Bart van der Wolf considers the best tool for capture sharpening, Focus Magic, it is greatly enhanced

This quote is very important, as it shows You insist on excessive usage of a tool that is creating or enhancing artifacts. Look at Focus Magic website and their examples. They are taking blurred images (that by definition have no artifacts), and end up with some artifacts in sharpened results (parrot, skating boy).

Set sharpening to a level that satisfy You and see if artifacts are visible at 200%. There is no point in additional sharpening and upscaling just to prove that “something is there”, especially when You use egressive, prone to errors algorithm.

PS
I don’t say Focus Magic is bad, or Mr. van der Wolf (whoever he is) doesn’t know photography. It just seems that it is an algorithm tuned more towards restoring blurred photos, rather than general tool for high quality photos. Different applications need different tools.


(Samuel Chia) #48

I think I made it pretty clear in my OP that I prefer Topaz Detail’s Deblur in that it avoids enhancing the diagonal ringing artifacts entirely. I’m not sure where the misunderstanding of myself forcing excessive usage of Focus Magic came about. I quote myself: ‘Note that I used Focus Magic at 300% amount to enhance the artifacts of the sharpening (sic: demosaicing) process, and I NEVER use such a high amount ever for capture sharpening, but the artifacts are still there, just less visible/amplified.’

Focus Magic is definitely one of the very best (and mature) tools out there that can help restore blurred signal in our digitally captured images. It is designed to deal with that, not just grossly out-of-focus images. It is actually not very good at dealing with motion blur, despite the creators’ claims. Bart’s recommendations are not to be taken lightly. His posts on the luminous-landscape forums, and elsewhere like on DPReview, Open Photography forums, are highly regarded and respected by the community. He has formidable experience and knowledge in a vast array of photographic science topics. Bart is not known to spout nonsense.

Often, the amount of sharpening that will avoid enhancing artifacts is too low to boost the lower contrast signals, so while the problems are averted, one is left with sub-optimal rendering of the lower contrast detail. For adequate sharpening of those, the already high contrast edges will tend to get clipped. One excellent work-around is to feather the sharpening at the ends of the tone scale, using Photoshop’s Blend-if sliders. That way it is possible to target both the low and high contrast edges. What was incredibly weird, is how Topaz’s method of deconvolution actually entirely avoid’s AMaZE’s ringing artifacts, but it did result in a strange plasticky look, similar to ‘Edges’ in RT.

What would be really nice to have I suppose, is AMaZE for essentially every part of a given image, and DCB’s rendering of high contrast diagonals. By minimizing the number of artifacts, as well as the severity of them, at the starting point, makes for more robust pixels to edit thereafter. Which means making a print of the same size that looks sharper, or being able to make even greater enlargements whist maintaining the same quality.


(Samuel Chia) #49

This is a brilliant example, thanks very much!

It is immediately very clear how the demosaicing algorithms fail and produce moire before auto CA correction, and how much better they perform after CA correction is applied. Thank you for helping me to see that.

AMaZE’s diagonal ringing artifacts are everywhere! LOL

One of my peeves with the higher-end Nikon and Sony cameras is the missing AA filter on the sensors. A well-tuned one is able to filter out the frequencies that will cause aliasing, to a large degree, making life so much easier during processing. With good deconvolution sharpening techniques, Bart has proved it is possible to restore almost all the original lost signal. What the AA filter can do, is impossible to replicate 100% in digital post processing.


(Ingo Weyrich) #50

I agree!
Here are some examples:

left is Amaze without raw auto-ca-correction, right is with raw auto-ca-correction

left is again Amaze without raw auto-ca-correction, right is DCB without raw auto-ca-correction

Left is DCB with DCB enhancement enabled, right is DCB with DCB enhancement disabled, both are without auto-ca-correction

In this case DCB enhancement (whatever it does) seems to introduce some artifacts while for other files it removes them…

And finally left Amaze, right DCB, both with raw auto-ca-correction

I can try it. I never really looked at the dcb code before. Is that the dcb_refinement ?

Here is the raw file I used for the screen shots.

Ingo

Edit: I forgot to mention that the examples above are without sharpening.


(Jacek Gozdz) #51

@heckflosse Great You are willing to help. I had a short discussion with Emil (when he was still active here). He wanted to implement parts of DCB (the enhancement) in AMAZE but we hitted the wall as we both suck at coding. This may work this time. :slight_smile:

Code I refere is indeed in refinemnt part you pointed. My code is quite well commented, an there is
// get rid of the overshooted pixels. It is a simple method, nothing fancy, but needs to be applied after green colors are interpolated. It uses diagonal pixels, and therefore will not work without changes in case of AMAZE stairsteps (they must be corrected using horizontal and vertical neighbours). I guess that is the place to put it amaze code.

That would be my start

As for dcb_enhancement - it takes map of interpolation directions created earlier by DCB, calculates local white balance, and then interpolates missing colors once again. If this is turned off (that was never my intention) the interpolation takes horizontal or vertical simple avarage creating blurred results.

PM me if needed (I will gladly participate)


(Ingo Weyrich) #52

@cuniek Thanks for the explanations and the pointer to the part of the amaze code. I’ll try it later today :slight_smile:


(Ilias Giarimis) #53

Jacek, can you give a more detailed opinion on this …
I mean …

  • What is the ideal WB which would help a demosaicer do a better job ?.
  • Would a variable WB help ? … I mean to not use WB multipliers calculated from all the frame but from a relativelly small tile around.

How do you explain the fact that especially on diagonals, Amaze amplifies Red and Blue while DCB looks to amplify Green ?.


(Jacek Gozdz) #54

@ilias_giarimis I was investigating how DCB works with different WB (using DCRAW code) and it turned out that settings that were the closest to light source temperature gave the best results. I was testing sunlight and bulb shots, nothing really exotic (no LED lights). I believe (that’s the right word) that WB that keeps all 3 channels unclipped is the optimal choice (tell me if this is similar to Your findings).

As for variable WB: I think it may work if there are highlights and shadows with different light source on the shot (in this case lumimance is the criteria). Didn’t test that.

I was experimenting with using different WB (very simple channel multiplier) on different areas of photo, but it worked only on monohue areas and failed miserabely on every single edge. I didn’t find method that set a good area boundary (edge detection) for my approach. After some time I quited.

As for different colors on diagonals: My method is using horizontal or vertical interpolation of green only, AMAZE is using R and B values to interpolate green diagonally. We also have different missing colors interpolation, and DCB is trying not to overshoot green pixels (if this happen, missing colors interpolation is giving overshooted red and blue). My method is less prone to CA, while AMAZE has other advantages.

I hope I helped.


(Andrew) #55

@samuelchia, thanks for the split files. Here are my thoughts, though this may take a while, new to Gimp…

Some comments. There’s the rule in digital sampling that the frequency being sampled must be less than half the sampling frequency. I know the anti-alias filter is there to enforce this, but I can’t help think that the “sampling frequency” here is 2x2 pixels and the sky-rock boundary is only a few pixels thick. Therefore there is always going to be noise on the boundary.

Your logic seems to be 1) there are unpleasant artifacts in the final image, 2) you remember there is possibly a weakness in amaze (diagonals), 3) Focus Magic highlights these, therefore 4) it must be amaze causing the problems. I suspect FM is the problem. @cuniek has alluded similarly. The product makes a big thing about deriving a car number id out of a blurred plate, and rescuing terrible exposures, but is it for fine art?

I’ve put your images in Gimp and compared them, but honestly, I’m not much the wiser (though obviously my untrained eye is not helping…). For others reading this thread, here are some screen grabs of one tiny section from the photo.

  1. “Bare” amaze (no chromatic ab. correction, no false colour correction steps). I see nothing wrong with this so far.
  2. Add in chrom.ab. and 1 false colour step. Now looking very reasonable.
  3. Apply unsharp mask, raduis 1, amount 200, everything sharpened with no halo control.
  4. Compare this with your FM result -

    (3) does not have the extreme colours and aggressive edge (though it does have an unwanted halo).
    The above is why I think the FM sharpening is the problem.

If you want top top quality perhaps a medium format sensor is needed?!

There is a cure for the edge of course. I expect you know this, but for anyone who doesn’t, one trick is to take the finished image and put a sharp version layer on top of the unsharpened image. Then with a small soft brush, trace over where sky meets land on the top layer and delete the sharp boundary. This gives a good result since it mimmicks the natural diffusion we see. Here is a larger part of the photo done this way -

Zoom in, I think it stands up quite well.
You will probably say it’s not for me to say how you process your prints, and that’s fair enough, but we all should be a bit practical and all this Amaze stuff seems a bit over the top when you have a very simple workaround.

@cuniek, I looked at your NEF file, but I honestly can’t see anything wrong! Yes there’s some purple fringing / aberration, but I can’t see these elusive artifacts (except for @heckflosse 's example). What is meant by stairsteps please? Obviously if you make a diagonal out of squares, it’s going to look like steps, beyond that I don’t get it.
Here is a bare amaze section and then with 1 step, chr.ab and Defringe in the colour tool (as you had).

Again, these sections are miniscule parts of the photo.
Where are the artifacts please?

I know little about demosaicing, I was just interested by this thread. Perhaps Amaze can be improved. But if you did something special for diagaonals, would it cause new problems in some other situation? Where do you stop? - special processing for corners, circles, squares, hair?!

Cheers


#56

That was my thought, too. Amaze (and Capture 1) reveals the weakness of Focus Magic, not the other way around. Focus Magic simply cannot handle an image as sharp as what Amaze puts out, it may be tailored to the softer ACR output.

Personally, in my shooting, I use Amaze with no sharpening at all. Perhaps if I used Adobe software then that wouldn’t be feasible, and that’s why an ecosystem of sharpening solutions is necessary.


(Samuel Chia) #57

That is completely the wrong thinking. There is no ‘possibility’ of a weakness in AMaZE’s diagonals. Emil has already confirmed that AMaZE does indeed produce diagonal ringing artifacts, and I showed to him the same examples as I showed here. There is no dispute there. I am not using Focus Magic to help myself see where the artifacts are, I use it to help others see where the artifacts are, and then tell them to see it again without sharpening, because that it much harder to see.

By avoiding artifacts as early as possible in the workflow, one is able to edit images more robustly, and enlarge them further than otherwise possible. I make large prints of my work, and I do not want to limit myself unnecessarily on how large I can go. The artifacts are not inherent to the camera, just how the data is interpreted (DCB does not have such artifacts, but has other problems). So it seems maybe this can be resolved.

There are so many flaws to this kind of thinking, it would be difficult to address each issue without digressing from this thread with several thousands of words.

The problem is not knowing where to look, and not knowing what to look out for. Here is a crop from Jacek’s DSC4042.NEF file, which clearly shows the diagonal ringing. No sharpening, all settings zeroed except for 2 steps of false color supression, which did not change the artifacts visibly but got rid of the offending and distracting false color artifacts. You should be able to very clearly see the diagonal ringing along the roof’s edge in AMaZE, which is completely not present in the DCB. I added a helpful line overlay below the AMaZE crop to highlight where and how long the ringing width is. DCB was with enhancement activated - DCB without enhancement is horrible and unusable.

https://drive.google.com/open?id=0B5AXKSbQEPuFNGg0cEc4UHlxenM

It is possible to see this in the crop I provide too, there is a green color on the edge of the DCB rendition. It is also on all the high contrast, thin edge details like in the fine branches you show. This is one of the issues I noted with DCB, when one has almost neutral-color edges. AMaZE gives the more correct color for the edge, especially in the roof edge (harder to say for sure in the fine branches). This green-bleeding artifact with DCB because of the way it interpolates green values (thanks for the explanation Jacek), is one reason I am not willing to switch to using DCB. I didn’t wish to cloud the already complicated thread with more issues. Already we have folks here (strangely) unwilling to accept Emil Martinec’s own admission that AMaZE does produce ringing diagonal artifacts and he wishes to resolve this if he can spare the time.


(Samuel Chia) #58

They are here, as highlighted with the white lines to show the approximate length of the ringing and the position. In the other fine structures you cannot see them in this crop because the detail is going about in all directions. See how the white line overlays are on an edge that is quite strongly diagonal?


(Samuel Chia) #59

Same here, though much harder to see. There is not so much ringing in this example as there are just diagonal artifacts where single pixels coalesce into single-pixel-width 45-degree strings.


(Ingo Weyrich) #60

I don’t think that folks are unwilling to accepts Emil’s admission about existing artifacts in Amaze. They are just not able to see them! Which does not mean they don’t exist! And which probably means they are not relevant for most of the folks here (including me, at least for my photo work). But as I already wrote in a pm to @cuniek we will try to improve the amaze output based on his knowledge of dcb code and my knowledge of amaze code.

Edit: My priority for this is less than my priority for pixelshift coding, so don’t expect any serious improvement during this week!