G'MIC exercises

I have not tried string manipulation, but that seem to be the only recourse. With ‘$=argpos’ and repeat, and appending string, this can be solved. Very ugly solution I must say.

One more for the heap; variance-weighted bilinear interpolation. That’s just a name I gave it - I don’t know the “official” name, assuming there is one. It has the property of making large edges clearer.

The idea is very simple: bilinear interp can be seen as a weighted sum of four pixel values, so we can easily add an extra weighting to that. The weighting is the reciprocal of variance of a 2x2 patch centred at each of the four pixels. The whole process effectively uses a 4x4 pixel input area for the output of one pixel.

This works best if the variance is averaged across all channels. The variance values can be computed quickly in advance with a box filter, but there’s no reason they can’t be done as part of the main loop either.

Here is an early reference version if anyone wants to experiment. I’d be very happy if the speed can be improved :slight_smile:

# Resize with variance-weighted bilinear scaling.
# format: gcd_resize_vwbl width,height
gcd_resize_vwbl : check "isnum(${1=0}) && $1>=1 && ${2=$1} && $2>=1"
  repeat $! l[$>]
    [0]x2 sqr.. boxfilter[-2,-1] 2 sqr. sub[-2,-1] max. 0 # 2x2 variance
    r. 100%,100%,100%,1,2 add. 1e-6 pow. -1 # average channels and invert
    $1,$2,1,{0,s} # The target resized image
    f. "begin(
        const interpolation=0;
        const boundary=1;
        const wr = (w#0-1) / max(w-1,1); # scaling ratios (small/large)
        const hr = (h#0-1) / max(h-1,1);
        px(x,y) = i(#0,x,y);     # point from source image
        vx(x,y) = i(#1,x,y,z,0); # point from variance image
      );

      X = x * wr;    # scaled coords
      Y = y * hr;

      xA = floor(X); # integer source coords
      xB = ceil(X);
      yA = floor(Y);
      yB = ceil(Y);

      lx = X-xA;     # usual ratios for lerp
      rx = 1-lx;
      ly = Y-yA;
      ry = 1-ly;

      w11 = rx * ry * vx(xA,yA); # variance adjusted weights for interp
      w12 = rx * ly * vx(xA,yB);
      w21 = lx * ry * vx(xB,yA);
      w22 = lx * ly * vx(xB,yB);
      S = w11 + w12 + w21 + w22;

      # final bilinear weighted sum
      (w11*px(xA,yA) + w12*px(xA,yB) + w21*px(xB,yA) + w22*px(xB,yB)) / S;
    " k.
  endl done

And an example of output - bicubic at the bottom for comparison:

3 Likes

Looks quite novel for a upscale filter. Maybe submit a paper with it since it’s yours and it’s unlikely anyone has came with it. I wish I knew of minmax sort idea years ago, but unlike that one, this one isn’t hard to come up with and someone did before me.

There’s almost always something prior - in any case, I’m no academic and wouldn’t know where to start. What I could do is a decent write-up of it (e.g. on the gmic github wiki).

It’s tremendously simple to implement. The boxfilter part can also be done using a separable convolution with (0.25,0.5,0.25). It would extend to 3 dimensions easily enough as well.

1 Like

Nothing new but when done well and explained clearly it might as well be. The exercise is all it is about.

Did you find a paper about it? I’d feel less uncomfortable doing a lengthy write up if the author gets proper credit! Plus, it would be nice to use the correct algorithm name.

I don’t remember. Usually people use names like your VWBL; won’t be much of an improvement on the naming front. :stuck_out_tongue: It may have come up during my investigation into resizing algorithms such as nohalo / lowhalo lohalo from GIMP (there are a few papers on that) or SAT-related that often survey and compare existing or state-of-the-art algorithms (many papers and graphics company slides on that dating way back).

1 Like

In that case I’ll hold off until it comes to light, especially if it could have a commercial source. At least I can say for certain nothing was copied intentionally!

I didn’t see anything that had copyright / patent written all over it. Do the normal open source thing and document your process and conclusions. I think that is sufficient.

Hmm, the open source way is usually to respect licences and give credit where it’s due. But, given this is a hazy memory of something you may have seen, I’m going to be pragmatic and just deal with that as it arrives (if it ever does). I’ll add a wiki page for the algorithm soon :slight_smile:

Edit: I added that to the g’mic community github wiki

2 Likes

Now, I know what one of you or David will say, but is there a way to get the dimension from the original image rather than preview image? My upcoming Hitomezashi GUI filter needs the original image dimension rather than preview dimension. Substitution rules shows nothing that can help.

Indeed, that’s probably not easily possible. So you’re probably stuck with letting the user choose the dimensions.

On the scaling algo above, it can be made much faster in g’mic by avoiding use of fill:

gcd_resize_vwbl :
  repeat $! l[$>]
    [0]x2 sqr.. boxfilter[-2,-1] 2 sqr. -[-2,-1] max. 0
    r. 100%,100%,100%,1,2 add. 1e-6 ^. -1
    mul.. . r $1,$2,100%,100%,3 /
  endl done

I added this to the bottom of the wiki page as well. Time to turn it into a more usable command…

2 Likes

What I meant is that people can come up with ideas independently. Document the thought process and if you discover there is prior art then acknowledge or credit accordingly. Nice Wiki entry and command. :+1:

1 Like

Ok, here’s my problem:

fx_rep_hitomezashi:
# ... snipped out code
u $bin_str_0,$bin_str_a,$bin_str_b,$seed_a,$seed_b

fx_rep_hitomezashi_preview:
ow,oh={w},{h}
skip "${5=}","${6=}","${7=}"

fx_rep_hitomezashi $* r. $ow,$oh,100%,100%,0,0,.5,.5
bin_str_0,bin_str_a,bin_str_b,seed_a,seed_b=${}
u "{$1}"\
"{$2}"\
"{$3}"\
"{$4}"\
"{"$bin_str_0"}"_{!$1?2}\
"{"$bin_str_a"}"_{$1?2}\
"{"$bin_str_b"}"_{$1?2}\
# ... snipped out code

If you were to read the code, I am sending arguments to ${}. However, if the inputs on ${5-7} contains space in them, then this no longer works. How do I fix it so that it works?

And debug doesn’t send out the offending lines.

I’d say you should use $"*" instead of $*.

I tried that, it didn’t seem to work.

But then, what is this 9th argument used in your code ? Have you defined a default value for $9 ?

I found the solution to my problem, see this commit - https://github.com/dtschump/gmic-community/commit/2d29309ccce6981b0c1ff69fbf0dc65d8d94a726 . The only downside is that it doesn’t allow ' char, but oh well to that, not a big deal.

After seeing a mention of pyramid based inpainting in a darktable highlights recovery thread

I thought I’d give it a try in g’mic. Some “insiders” have seen a version already, but it had a problem: upscaling with the resize command in g’mic - in bilinear mode - is not pixel aligned with a downscale. I’ve hopefully solved this by using a 2x2 box filter (that’s the convolve part) after nearest-neighbour scaling.

gcd_pyramid_inpaint : check ${"is_image_arg $1"}
  pass$1 0 ge. 1 store. msk
  repeat $! l[$>]
    wh={[w,h]-1}
    L={ceil(log(max(w,h))/log(2))} # find required number of levels
    i $msk z 0,0,{D=2^$L;[D,D]-1},1 # force 2^n dimensions
    eq. 0 mul.. .
    repeat $L I,M={[$>*2,$>*2+1]}
      +r[$I,$M] 50%,50%,100%,100%,2
      +eq. 0 add. .. div[-3,-1] gt. 0
    done
    i[0] (0.0625,0.125,0.0625;0.125,0.25,0.125;0.0625,0.125,0.0625)
    repeat $L
      l[0,-4--1] rm.
        r. [1],[1],[1],[1],1 convolve. [0]
        eq[2] 0 j[1] .,0,0,0,0,1,[2] rm.
      endl
    done
    rm[0,-1] z 0,0,$wh # restore dimensions
  endl done

gmic sp leaf 100%,100%,100%,1 ellipse. 50%,50%,"{[w/4,h/3]}",0,1,1 negate. gcd_pyramid_inpaint.. .
leafinpaint

On this laptop, it takes 0.072s

2 Likes

This looks actually pretty close to what I’ve done with my inpaint_pde command, the one I use to reconstruct 3D color luts from a set of keypoints.