New G'MIC filter: DCCI 2x resize (updated)

Software G'MIC
1

Update - Thanks to David it’s now been included in G’MIC stdlib:

  • Now moved to G’MIC stdlib, under Repair > Upscale [dcci2x]
  • Associated G’MIC command -scale_dcci2x
  • See post further down for more info/screenshots

original post:

Available in testing > garagecoder > Dcci 2x resize

Hi folks

Stumbled upon an interesting article (well, to me anyway!) on wikipedia - Directional Cubic Convolution Interpolation - Wikipedia - and couldn’t help but try it out.

Unlike many 2x upscale algo’s it’s aimed at general images rather than pixel art and seems to avoid “stair-stepping” artifacts without too much visible smoothing.

There are a few changes to how the algorithm is performed to suit G’MIC, it’s also been extended to allow arbitrary channels (so can do colour/opacity) and use neumann boundary. Beware there could well be errors since I haven’t checked it over properly yet!

1 Like
2

Just tried it (not showing). Impressive. Ran it twice and each time no stairstepping; a noticeable chromatic aberration on the image I tried it on, but I can work with that. Thanks a heap GC. :slight_smile:

3

Thanks for the feedback Lyle, for now it merely scales each channel independently. If there happens to be a strong edge in one colour channel but not the others it will only be smoothed in that one channel. There are a few easy ways to change that if desired, but further testing is needed to see which produces the most ideal output.

4

The latest update adds cross-channel edge detection, which might help with your chroma issue and should improve output quality.

Technical: it turns out just averaging the gradient sums is enough to improve PSNR for the tests I did.

1 Like
5

Cool stuff again GC. Even with the issues, the results were impressive to me. I’ve run across many uprez techniques over the 15 years that I’ve bothered to research the topic and I can tell you I’ve never ran across a better one than this method. Just my opinion for what it’s worth. :smile:

1 Like
6

Good news Andy!
Thanks, I didn’t know this method. Maybe I’ll take some time to see if this can be re-implemented just by a mathematical expression (could be run in a multi-threaded way).
I’ll let you know !

7

Yes please do, the current one is a bit of a hack to see how good the output was.

8

This sounds like a good filter to quickly add into PhotoFlow, for image up-scaling!

Since in my case the filter will be parallelized and applied to image tiles, I would at some point need to know the precise tile padding required by this algorithm. Any idea?

Also, would you have any good test images to share, so that I can compare the result with the default “no halo” method used in photoflow?

Thanks for your great work, Andy!

9

Thanks Andy, here is my version. I’ve done a straightforward implementation of the algorithm presented in the link you gave.

Feel free to give feedback !

10

Ah, it seems I have some issues on the right/down boundaries.

EDIT : OK this one should be better : http://pastebin.com/ijbWWZBf

11

Aha, that’s probably how I should have prototyped it in the first place! Now the two can be compared, I notice there are very small differences in the output - could be rounding errors or simply me making mistakes but at least I can check it. Since the math parser version seems slower I wonder if the two can be combined?

I didn’t distribute the interpolated pixels until the end, which means the kernel is smaller but it makes it harder to get my head around.

@Carmelo: I think it would need overlap of 2 pixels (possibly 3) since the largest kernel I use is 5. As for test images I can maybe do some later, otherwise there are some b&w examples linked on the wiki or you can test it with gimp + g’mic.

12

Andy, I’ve done that primarily to test the robustness of the new features of the math parser.
If your version is faster, then we probably should stick with it instead.
That would be cool anyway to obtain the same results, as well as the same image size at the end
(i.e. 2width-1 x 2height-1). But of course, I’ll vote for the fastest version!

13

Cool, my task now is to compare output for correctness and tidy up the code. I have some ideas where differences might happen due to precision errors; it’s possibly where I optimised the math for fewer calculations but speed loss should be minimal if I have to change it. Luckily I have some time now :smile:

14

Don’t worry, if you do not have something ready at hand I’ll take some of my images and do some experimenting. Just wanted to see if you had examples that clearly show the possible artifacts… probably some sharp shot of tree branches might be a good candidate to see the staircase artifacts?

15

There might be an issue with the math parser version - it almost always has worse PSNR results than mine even in greyscale so I did a comparison; there are large differences within a 3 pixel boundary around the edges.

Repeating the tests after removing a 3px border for all images shows the math parser with better PSNR, but only a very small amount (~ 3x10^-6). With colour images it still does worse due to not averaging gradients sums across channels.

Edit: suspect the smoothing step in the wiki article may have the pixel values the wrong way round, need to compare properly with the paper / matlab example.

16

Can confirm the matlab source code doesn’t seem to match the wiki page as far as the “Smooth Area” sections are concerned (i.e. the wiki appears to be wrong). The weights match up but the interpolated pixel values are the wrong way around.

With that patched the k exponent actually affects output, I might have to add a slider for it since there’s more edge smoothing overall now. For now I’ve just patched the filter as it stands… will have to leave the rest for later.

17

Ah, that’s interesting !
Maybe I’m asking too much, but do you think the fix could be simple using the math parser instead (my version).
I’ll be interested to understand what differences it adds.
Anyway, I’m interested to put your faster version directly in the g’mic stdlib, if you agree with that idea.
Does it work when you have an arbitrary number N of channels ? Does it work for 3d volumetric images ? (i.e. just double the width and height of all planes ?).
Thanks!

18

The fix is very simple for both methods, if we take a sample from the math parser (I’ve omitted the boundary for brevity):

 DownRightPixel = (-j(-3,-3) + 9*j(-1,-1) + 9*j(1,1) - j(3,3))/16;
 UpRightPixel = (-j(3,-3) + 9*j(1,-1) + 9*j(-1,1) - j(-3,3))/16;
 DownRightPixel*weight1 + UpRightPixel*weight2

The quickest way is to switch the numbers of weight1 and weight2 but more correctly it should read:

 DownRightPixel = (-j(-3,-3) + 9*j(-1,-1) + 9*j(1,1) - j(3,3))/16;
 UpRightPixel = (-j(3,-3) + 9*j(1,-1) + 9*j(-1,1) - j(-3,3))/16;
 UpRightPixel*weight1 + DownRightPixel*weight2

Where we take “UpRightPixel” to be the 45 degree interpolated value. The same needs to be done for the horizontal/vertical pass. In the other method it’s a case of referencing two images the other way around so every bit as easy.

Arbitrary channels is easy enough, in fact only support for opacity changed that. What I should really do is have a “native” command which doesn’t care about channels/opacity, then for the g’mic filter split the opacity and call the native command on those. That has to be done because of gradient averaging across channels.

3d volumetric :grinning:
Hahah ok well without alterations it would work as is but with no interpolation between “slices”. Assuming you do a third pass in the depth plane it would be interpolating between previously interpolated values… it needs some thought!

Adding to g’mic stdlib is a great idea, but I still have cleanup to do. For instance I do a third unnecessary gradient amongst other things, it just made it much simpler to deal with in my head. If you fancy doing the tidy up for me though I’m more than happy :smile:

19

I forgot to answer about the changes the fix produces! It’s mostly quite subtle but here’s a list:

More smoothing in general. Previously higher threshold gave more noticeable jaggies, due to the “Smooth Area” pixels being in the opposite direction from the edge.

Exponent k has more effect. It’s actually quite difficult now to see staircase without reducing k.

PSNR is improved. This is something I’ve never tried before so could well make mistakes; scale to 50% by nearest-neighbour and scale back up then compare after crop (since I now include the 1px removal) using g’mic -psnr. This is what made me spot it - decreasing k actually improved PSNR which made no sense.

Edit: I’d really like somebody to check what I’m saying! Especially compare wikipedia with the original paper/matlab source.

1 Like
20

I hope somebody can confirm this, but I now think the wiki page has another issue… the description of the horizontal/vertical gradient seems to be wrong.

The original paper appears to mention a 5x5 kernel with 7 absolute gradients summed in each direction. The matlab source also shows 7 subtractions per direction, whereas the wiki shows 9 in a diamond shape (within a 7x7 kernel).

This might explain some issues I’ve noticed: PSNR for the G’MIC filter being best around threshold 1.5, comparison with matlab test output images being ever so slightly different and there being quite a lot of edge smoothing in general.

It’s probably best to stick to the paper so I’m going to compare both (i.e. redo the filter).