Attempt at cubism

After taking an extremely long break I need some help with a filter I want to make. The existing cubism one can only handle predetermined shapes layered on top of each other and doesn’t really look all that cubist, so I thought I’d make one that uses segmentation to cut up and reorganise the image. So far it looks like this:

#@gui Cubism: fx_jr_cubism_preview
#@gui : sep = separator()
#@gui : 0. Recompute = button()
#@gui : 1. Threshold=float(2,0,15)
#@gui : 2. Smoothness=float(1,0,500)
#@gui : 3. Amplitude=float(10,0,100)
fx_jr_cubism:
+to_rgb b. $2 segment_watershed. $1 # add energy polygonize option
f. ">begin(rand=u(100,200));srand(i*rand);u"
sh. 0 n. 0,1 rm.
sh. 1 n. 0,1 rm.
sh. 2 n. 0,{2*pi} rm.
amp={$3*w*0.01}
f.. "val=(I(#1));
xoff=val[0]*"$amp";
yoff=val[1]*"$amp";
ang=val[2];
I(x*cos(ang)+y*sin(ang)+val[0],y*cos(ang)-x*sin(ang)+val[1],0,3,3)"
rm.
fx_jr_cubism_preview:
fx_jr_cubism ${2-4}

It takes segmented copies of images and uses the copies to generate maps with the same segment shapes but pseudorandomised values. (Thank goodness for srand()!) The maps are used not only to offset parts of the original image but also to rotate them by any angle. If I add a bit of code I can get more control over the angle.

Now I have two problems:

  1. anti-aliasing. Like with the kaleidoscope layer cake, I don’t want to take the brutal route and temporarily upscale the image.

  2. rotating from the centre of each segment. How would I find the centre of each segment and verify that a pixel belongs to a certain segment?

Combining the two problems makes this even more difficult since I would need to slightly blur the boundaries between the segments, which would make the second problem harder.

I wanted to avoid repurposing the patchmatch resynthesise thing which already has a blending size parameter which can be used for anti-aliasing purposes, but the second problem would still exist and I don’t know how I’d rotate the image within each patch.

Edit: so my first idea is to scan for patches of the map which have different values. I’ll use a fourth channel for the map to filter out which patches have already been scanned

Edit 2: is there anything which will dump the x and y coordinates of all pixels with a certain value? Also how does rot() work if there’s no difference between 1D and 2D vectors?

That’s why I use rot_x(a,b), and rot_y(a,b) on my commands. That being said, here’s the way to get center of each segment. Antialiasing can be done with making antialias(xp,yp)=(i(xp+0)…i(xp+v))/4, and finally insert it at the end of f block.

rep_average_coordinate_of_value_n 0
rep_average_coordinate_of_value_n:
channels 0
label
use_zero=$1
initsize={iM}
initsize-={$use_zero}
vsize={$initsize*2}
eval ${-math_lib}"
coordinates_per_val=vector"$vsize"(0);
count_per_val=vector"$initsize"(0);
for(px=0,px<w,px++,
    for(py=0,py<h,py++,
        pos=i(#0,px,py,0,0);
        coordinates_per_val[pos*2]+=px;
        coordinates_per_val[pos*2+1]+=py;
        count_per_val[pos]++;
    );
);
for(n=0,n<"$initsize",n++,
    coordinates_per_val[n*2]/=count_per_val[n];
    coordinates_per_val[n*2+1]/=count_per_val[n];
);
coordinates_per_val;
"

Then if you need to access the values found on that. Use ${}.
That means you may need to modify your command to adapt the code above.

Side note: If you’re going to push patch, let me know. Also, I’ll PM you later about the superstreak, it’s possible to have randomize as a button. See this sample from David.

#@gui Foo random : rand_foo, rand_foo_preview
#@gui : Param1 = float(0,-100,100)
#@gui : Param2 = float(0,0,3)
#@gui : Randomize = button()
rand_foo :
  * $2 + $1 c 0,255

rand_foo_preview :
  p1,p2=$1,$2
  if $3 p1,p2={[u(-100,100),u(0,3)]} fi
  rand_foo $p1,$p2
  u "{"$p1"}{"$p2"}{0}"
1 Like

Thanks for sharing this! You’ve indirectly introduced me to the label command which is really useful, though I wish there was a version of it which works for multiple channels and outputs a single channel instead of the one we have now which works on individual channels independently. I’ll try and repurpose what you’ve built.

That is indeed a pertinent remark. Next version of G’MIC (2.9.2) will change the behavior of label to work on multi-channels images and output only a single-channel image.

2 Likes

That’s problem number two sorted, namely rotating around the centres of each thing. Until 2.9.2 releases there’s a temporary workaround I’ve found for the label command which I’ll remove later.

#@gui Cubism: fx_jr_cubism_preview
#@gui : sep = separator()
#@gui : 0. Recompute = button()
#@gui : 1. Threshold=float(2,0,15)
#@gui : 2. Smoothness=float(1,0,500)
#@gui : 3. Offset Amplitude=float(10,0,100)
fx_jr_cubism:
ww={w}
hh={h}
+b. $2 segment_watershed. $1
f. ">begin(srand();rand=u(100,200));srand(i*rand);u"
repeat {s}
sh. $> *. {u(1,2)} rm.
done
to_gray.
label.
initsize={1+iM#1}
$initsize,2,1,1
vsize={$initsize*2}
eval ${-math_lib}"
coordinates_per_val=vector"$vsize"(0);
count_per_val=vector"$initsize"(0);
for(px=0,px<w#1,px++,
    for(py=0,py<h#1,py++,
        pos=i(#1,px,py);
        coordinates_per_val[pos*2]+=px;
        coordinates_per_val[pos*2+1]+=py;
        count_per_val[pos]++;
    );
);
for(n=0,n<"$initsize",n++,
    coordinates_per_val[n*2]/=count_per_val[n];
    coordinates_per_val[n*2+1]/=count_per_val[n];
	I(n,0)=coordinates_per_val[n*2];
	I(n,1)=coordinates_per_val[n*2+1];
);
"
$ww,$hh,1,3
repeat {3}
sh. $> f. ">begin(srand();randm=u(100,200);randa=u(1,3));srand((i#1+randa)*randm);u" rm.
done
sh. 0 n. 0,1 rm.
sh. 1 n. 0,1 rm.
sh. 2 n. 0,{2*pi} rm.
amp={$3*norm(w,h)*0.01}
f[0] "begin(amp="$amp");val=(I(#3));
xoff=val[0]*amp;
yoff=val[1]*amp;
ang=val[2];
xro=i(#2,i(#1,x,y,0,0,1),0,0,0,1);
yro=i(#2,i(#1,x,y,0,0,1),1,0,0,1);
vx=x-xro+xoff;
vy=y-yro+yoff;
I(vx*cos(ang)+vy*sin(ang)+val[0]+xro,vy*cos(ang)-vx*sin(ang)+val[1]+yro,0,3,3)"
rm[^0]
fx_jr_cubism_preview:
fx_jr_cubism ${2-4}

Using this I can turn this photo from Pexels into this:

@Reptorian I’m not clear on how the antialiasing procedure you’re describing works. The way I’m thinking of doing it involves getting 2x2 matrices where some of the pixels act as if they’re in different cells and bilinearly interpolating. How do I turn everything everything after begin() in the last f block into a macro to do this?

When I’m done with this I’ll add an arbitrary map option and I’ll try making a border option.

I tried making a macro and it worked, but I have no idea how I’ll be able to use it since averaging multiple pixel values at the end like this will only make the image blurry.

f[0] "begin(amp="$amp");
cubism(xx,yy)=(
val=(I(#3,xx,yy,0,c,1));
xoff=val[0]*amp;
yoff=val[1]*amp;
ang=val[2];
xro=i(#2,i(#1,xx,yy,0,0,1),0,0,0,1);
yro=i(#2,i(#1,xx,yy,0,0,1),1,0,0,1);
vx=xx-xro-xoff;
vy=yy-yro-yoff;
I(vx*cos(ang)+vy*sin(ang)+val[0]+xro,vy*cos(ang)-vx*sin(ang)+val[1]+yro,0,3,3));
cubism(x,y)"

The avg() should not mix xp+1 and/or yp+1, but rather xp+.5, and yp+5. If you think about upscaling nearest neighborhood, every pixels that are created are essentially .5 ahead of the new pixel. That being said, if that what works, then I think this filter is going somewhere. If all fails then, I would use r2dx 200%, then resize it back.

It didn’t work. I will have to use a last resort for now.

Edit: so the real problem is that I’m not dealing with functions like in the KLC script, but rather discrete bitmaps since label produces bitmaps. If I’m gonna avoid supersampling then I would need to generatefunctions for parts of the bitmaps or even take a morphological route. I have no idea how I’d do that.

That’s something way over my head, I don’t think I can help you avoid the last resort.

Perhaps you could glean from what others have done. E.g.

https://gmic.eu/reference.shtml#cubism

– From random web search.

1 Like

Seeing smoothstep() in there made me think - I could somehow use wavelets to sort this out…

Yes. I’ve known about this family of functions for a while now.

What I’m currently thinking of involves shifting the label image by 1 pixel in eight directions and interpolating between the final values of each corresponding pixel in what will be the result image by shifting it by 0.5 pixels in the opposite directions, factoring into account the difference between the pixels since we’re dealing with labelled features. If it was just the single direction to the right, for example, then if the current pixel was for segment 6 and a pixel for segment 8 was to its immediate right, I would shift the label image to the right by 0.5 with linear interpolation on and then do some averaging or something before moving it back.

Would that be processing intensive? How large of an area are we talking about?

@Joan_Rake1, I don’t want to rush you, but I am curious, how is it coming?

No progress so far. I’ve been busy with other things and even just thinking about this makes me want to procrastinate. Programming anything has been a hair-straining bugbear of mine for a while and I’ve avoided it for so long because I just wanted something else to do.

Yeah, that’s understandable. Personally, I really dislike programming in c++ myself even though that skill is needed for me to finally be happy with open-source software as I’m not content with either GIMP or Krita 100% yet (one lacks nde, and the other lacks filters+selection, frankly neither are there yet). That being said, I hope you find something else. Regarding g’mic programming, my goal to translate just about every single pdn plugins into g’mic-qt is becoming a reality. I even have some sections done for the even more complicated one with no permission to reverse engineer such as Red Ochre’s works which involves rather meticulous coding.

I am super super stupid! I was working on the pixel values at the end when I should really also be using the segment ids. I need to figure out how I find the number of pixels with a certain value in a 3x3 grid and then use those for a weighted calculation involving the final values. I could even shave it down to a 2x2 grid if I know what I’m doing.

1 Like

That sounds a lot less computationally intensive, and better optimized.

Do you mean dividing the image into 9 sections and then counting the pixels of a certain value?

No, I mean that I’d have a 3x3 grid centred on the current pixel, find the number of pixels for each label id within that grid and then run a calculation which will interpolate between what the final values the centre pixel would be if it had any of those ids.

I could also repurpose this technique to generate images which highlight the edges between the segments.