Reptorian G'MIC Filters

Ok, I’d like to know what the hell is going on here because it sure ain’t in the code!

It sometimes output to black, other times it works.

The code in question:

#@cli rep_lavander_binary_map: _size_of_arr_by_power_of_two>0,_mode={ -1=custom | 0=and | 1=or | 2=xor },_bin_a,_bin_b,_shift>0,custom_expression
#@cli : Create a texture that does the following steps:
#@cli : 1) Create an array of integer that matches index with size of 2^n.
#@cli : 2) Create another array which is the count of 'bin_a' in binary representation of the previous array.
#@cli : 3) Count sort the array of integer with the count of binary number array.
#@cli : 4) Delete all other array other than the sorted array.
#@cli : 5) Generate a surface with row and column the same size as the primary array, and use formula which takes values from the sorted value, and then find the count of 'bin_b' in binary representation of the found value.
#@cli : - Built-in Macros -
#@cli : flip(v) - Flips the bitwise values of number
#@cli : and(a,b) - a & b
#@cli : or(a,b) - a | b
#@cli : - End of Built-in Macros -
#@cli : Note : The idea comes from Lavander at discord/generative. Based on a Python code.
#@cli : Default values: '_size_of_arr_by_power_of_two=11','_mode=2','_bin_a=01','_bin_b','_shift=0',
+new_rep_lavander_binary_map:
skip ${1=11},${2=2},${3=01},${4=10},${5=0},${6=}

check "$1>0&&($1==int($1))&&(isint($2)||same('-','$2'))"

a6={size('$6')}
shift,length,mode:=int(abs($5)),1<<$1,v=same('-','$2');n=isint($2)?$2:0;v?0:(n>=0?n%3+1:-((abs(n)-1)%3+1))
size_of_arr={int(log2($length+$shift-1))+1}
str_code=n=init_num()
act_arr_sortclass={$mode>-1}

if !$mode&&!$a6 error custom_formula_required fi

$length,1,1,{1+$act_arr_sortclass},>"begin(
         const first_binary_to_search=0b$3;
         const second_binary_to_search=0b$4;
         const size_of_first_binary_to_search=size('$3');
         const size_of_second_binary_to_search=size('$4');
        );
        count=0;
        current_value=x;
        current_number_of_binary_digits_per_value=(i?int(log2(current_value)))+1;
        bit_mask=(1<<current_number_of_binary_digits_per_value)-1;
        c?(
         binary_to_search=second_binary_to_search;
         size_of_binary_to_search=size_of_second_binary_to_search;
        ):(
         binary_to_search=first_binary_to_search;
         size_of_binary_to_search=size_of_first_binary_to_search;
        );
        while(current_number_of_binary_digits_per_value>=size_of_binary_to_search,
         test_section=current_value>>(current_number_of_binary_digits_per_value-size_of_binary_to_search);
         test_section==binary_to_search?(
          ++count;
          bit_mask>>=size_of_binary_to_search;
          current_number_of_binary_digits_per_value-=size_of_binary_to_search;
         ):(
          bit_mask>>=1;
          --current_number_of_binary_digits_per_value;
         );
         current_value&=bit_mask;
        );
        count;
        "
 
if $act_arr_sortclass
 
 $length,1,1,1,x rv[-2,-1]
 
 +channels. 0 # Create copy as we will preserve the the generated output as lookup table

 if $shift -. {im} fi

 # Pixel sort integer by count sort
 +histogram. 100%
 f[-1] >begin(n=0;);i?(v=n;n+=i;v;);
 f[-2] >v=i(#-1,i);++i(#-1,i);v;
 {w#-2},1,1,1
 eval[-3] i(#-1,i)=i(#-5,x);
 rm[-5,-3,-2]
 a[-2,-1] c
 
fi

if !$mode str_code..=ix=max_index-x;iy=max_index-y; fi

binary_set_to_search,size_of_binary:=$act_arr_sortclass?[size('$4'),0b$4]:[size('$3'),0b$3]

$length,$length,1,1,:"begin(
  const shift_factor=$shift;
  const mode=$mode;
  const binary_set_to_search=$binary_set_to_search;
  const size_of_binary_set=$size_of_binary;
  const reference_channel=1+$act_arr_sortclass;
  const lookup_table_pos=$act_arr_sortclass;
  flip(v)=xor((1<<(int(log2(v))+1))-1,v);
  and(a,b)=a&b;
  or(a,b)=a|b;
  x()=i(#-1,x,0,0,reference_channel,2);
  y()=i(#-1,y,0,0,reference_channel,2);
  ix()=i(#-1,max_index-x,0,0,reference_channel,2);
  iy()=i(#-1,max_index-y,0,0,reference_channel,2);
  mode==3?(
   init_num()=xor(i(#-1,x,0,0,reference_channel),i(#-1,y,0,0,reference_channel));
  ):
  mode==2?(
   init_num()=i(#-1,x,0,0,reference_channel)|i(#-1,y,0,0,reference_channel);
  ):
  mode==1?(
   init_num()=i(#-1,x,0,0,reference_channel)&i(#-1,y,0,0,reference_channel);
  ):
  mode==-1?(
   shift_factor?(
    init_num()=(x+shift_factor)&(y+shift_factor);
   ):(
    init_num()=x&y;
   );
  ):
  mode==-2?(
   shift_factor?(
    init_num()=(x+shift_factor)|(y+shift_factor);
   ):(
    init_num()=x|y;
   );
  ):
  mode==-3?(
   shift_factor?(
    init_num()=xor(x+shift_factor,y+shift_factor);
   ):(
    init_num()=xor(x,y);
   );
  ):(
   init_num()=int("$6");
  );
 );
 count=0;
 "$str_code";
  count=0;
  n=abs(n);
  !inrange(n,0,w,1,0)?(
   num_of_binary_digits_of_n=(n?int(log2(n)))+1;
   bit_mask=(1<<num_of_binary_digits_of_n)-1;
   while(num_of_binary_digits_of_n>=size_of_binary_set,
    test_section=n>>(num_of_binary_digits_of_n-size_of_binary_set);
    test_section==binary_set_to_search?(
     ++count;
     bit_mask>>=size_of_binary_set;
     num_of_binary_digits_of_n-=size_of_binary_set;
    ):(
     bit_mask>>=1;
     --num_of_binary_digits_of_n;
    );
    n&=bit_mask;
   );
   count;
  ):(i(#-1,n,0,0,lookup_table_pos));"

rm..

if !$act_arr_sortclass&&$shift -. {im} fi

Even worse, the GUI filter is all messed up with this new code. Just remove new_ to find that out!

EDIT:

Seems that it has to do with the image before $length,$length,1,1 part. I will investigate it, but it is a weird issue. Ok, I can trace it down to the $length,1,1,{1+$act_arr_sortclass}, but I see no reason why this should happen. So, I don’t know why, so there is something really wrong here.

1 Like

I don’t have this problem on Linux.
Maybe it’s just display playing tricks on you? Just a shot in the dark, but have you tried moving the image around, or zooming, when it’s black? Or even resizing the window? :shushing_face:
It’s also almost pitch black when i turn off normalization.

Forget it, i see you have only zero’d pixels in your video.
Still, i can’t get it to fail.

Maybe it’s a OS-specific thing. I traced it down to a single variable which is current_number_of_binary_digits_per_value.

On this block: $length,1,1,{1+$act_arr_sortclass} : Add this line of code.

if(x==128,print(current_number_of_binary_digits_per_value));

My results:

C:\Windows\System32>gmic +new_rep_lavander_binary_map 8,0
[gmic]./ Start G'MIC interpreter (v.3.3.6).
[gmic_math_parser] current_number_of_binary_digits_per_value = (uninitialized) (mem[45]: scalar)
[gmic_math_parser] current_number_of_binary_digits_per_value = 8
[gmic_math_parser] current_number_of_binary_digits_per_value = 8
4::MSB1aW50OCBsaXR0bGVfZW5kaWFuCjI1NiAxIDEgMiAjODIKeJydTkkOACEIg9b/v3mmYg96IFET04WyRPwv9TONCfFdJwD7XV0Iiu/5135rkLTfzb/eH4vOUKGip64jyu/qQp165l/7rclB+9382/0fpZ4DAg==
[gmic]./ Display image [0] = '[begin( const shift_factor=$s...'.
[0] = '[begin( const shift_factor=$shift; const mode=$mode; const bi...':
  size = (256,256,1,1) [256 Kio of float32].
  data = (0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, ... ,1,1,1,2,2,2,2,3,3,3,2,2,2,3,1,1,1,2,2,2,1,2,2,2,2,2,2,3,3,3,2,2,2,3,2,1,1,1,2,2,2,2,3,3,3,2,2,2,3,2,1,1,1,2,3,3,3,2,1,1,1,2,3,3).
  min = 0, max = 4, mean = 1.3125, std = 0.802833, coords_min = (0,0,0,0), coords_max = (251,35,0,0).
[gmic]./ End G'MIC interpreter.

C:\Windows\System32>gmic +new_rep_lavander_binary_map 8,0
[gmic]./ Start G'MIC interpreter (v.3.3.6).
[gmic_math_parser] current_number_of_binary_digits_per_value = (uninitialized) (mem[45]: scalar)
[gmic_math_parser] current_number_of_binary_digits_per_value = 1
[gmic_math_parser] current_number_of_binary_digits_per_value = 1
0::MSB1aW50OCBsaXR0bGVfZW5kaWFuCjI1NiAxIDEgMiAjMTQKeJxjYBgFIxkAAAIAAAE=
[gmic]./ Display image [0] = '[begin( const shift_factor=$s...'.
[0] = '[begin( const shift_factor=$shift; const mode=$mode; const bi...':
  size = (256,256,1,1) [256 Kio of float32].
  data = (0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, ... ,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0).
  min = 0, max = 0, mean = 0, std = 0, coords_min = (0,0,0,0), coords_max = (0,0,0,0).

@David_Tschumperle Any explanations on this perplexing bug? It’s perplexing because:

        current_value=x;
        current_number_of_binary_digits_per_value=(i?int(log2(current_value)))+1;

When x is 128, it should be 8. Not 1. So, there should be no reason why I get a 1 here randomly. It is usually 8 though. With multiple computers, this bug does show up on Windows. That’s for certain.

Could you please write a minimal code to reproduce the issue ?
Unfortunately, I can’t spend hours understanding a large chunk of code to find a possible bug.
Thanks!

I fixed it by separating x part and using i on fill instead. I"ll try to find a minimal example, but hopefully I can find that because this is the only time I have ever seen this issue.

After which line exactly?
Placed it under this:
current_number_of_binary_digits_per_value=(i?int(log2(current_value)))+1;
But it doesn’t print anything.

Ok, leave it to me to find the minimal example. I pushed the new Lavander Binary Map after I fixed bugs.

Old one with using 11 (2048x2048) : 0.667 s
New one with using 11 (2048x2048) : 0.057 s

~12x faster! It’s using lookup table now. That’s all the practical changes.

I’m just gonna save this here and bookmark it myself - https://cs.wellesley.edu/~pmetaxas/ei99.pdf

Apparently, it’s possible to parallelize dithering effect.

Now, I’m releasing a new version of Color Harmonies for testing:

So, if you’re able to test it, import this via G’MIC plugin, and then let me know how it worked out.

Also, about the earlier message, I wanted to know if value() is just invisible text(). And isn’t there an alternative name because value() seems to be giving the impression it’s just number.

To which post are you referring?

Recently deleted post.

Here on Linux, it seems to work as expected.
Not sure what changed from the old version (besides the points size, and the coords box missing), but it works :wink:

Also tried the CLI version :

Looks pretty useful for color amputees like me :slight_smile:

The result shouldn’t change.

However, the behind the scenes changes:

  1. Usage of subcommands. I think it is easier to read.
  2. More interactive points. The old one has only one interactive point. There’s a plus icon next to point, so I hidden all the coordinates.

When I figure out value(), maybe less variables too.

Just one bug, there’s a occasional spasm out problem with moving interactive points. I don’t know why it happens. EDIT: My changes seems to reduce the spasm issue. And this besides, I updated the Color Harmonies filter.

There seems to be a bug exporting though. I’ll look into that. Oh all of my file export script seems to delete the original images. I fixed it by adding a + next to it, but not sure if that is what I was suppose to be doing with export scripts.

@grosgood After a conversion with the person that made the Python code we were discussing on discord, I finally have my own solution:

#@cli rep_scale_upscaled_pixelart_into_original:
#@cli : Automatically scale pixel art work that has been upscaled into its original dimension.
rep_scale_upscaled_pixelart_into_original:
 foreach {
  ss={s}
  +compose_channels + /. $ss
  +fft[-1] 
  a[-2,-1] c 100%,100%,100%,1,cabs(I#-1) => fft_amplitude
  keep[0,-1]
  crop[fft_amplitude] 0,0,{([w,h]-1)>>1}
  +resize[fft_amplitude] {w#$fft_amplitude},1,100%,100%,2
  +resize[fft_amplitude] 1,{h#$fft_amplitude},100%,100%,2
  new_dimensions={[xm#-2,ym#-1]}
  keep[0]
  resize $new_dimensions,100%,100%,2
 }

The sad thing is that FFT is faster on numpy than on G’MIC. Significantly faster, but the rest is fine. Anyways, here’s the result with a test:

Alternative code that works on harsher compression:

#@cli rep_scale_upscaled_pixelart_into_original:
#@cli : Automatically scale pixel art work that has been upscaled into its original dimension.
rep_scale_upscaled_pixelart_into_original_2:
 foreach {
  ss={s}
  +compose_channels + /. $ss
  +fft[-1] 
  a[-2,-1] c 100%,100%,100%,1,cabs(I#-1) => fft_amplitude
  keep[0,-1]
  s[-1] x,2 mirror[-1] x
  s[-2,-1] y,2 mirror[-3,-1] y
  add[-4--1] crop. 0,0,66%,66%
  +resize[fft_amplitude] {w#$fft_amplitude},1,100%,100%,2
  +resize[fft_amplitude] 1,{h#$fft_amplitude},100%,100%,2
  new_dimensions={[xm#-2,ym#-1]}
  keep[0]
  resize $new_dimensions,100%,100%,2
 }

New CLI Filter rep_blend_by_statistical_mode_8bit_layers though it serves a purpose with one other filter, it’s much faster for 8-bit images.

Basically, it calculates statistical modes of colors.

See example here:

Note the last image does not have track outline.

I don’t understand that sentence, but looking at the images i suppose it calculates the similitudes between all images, so it can recover the background beneath the tracks? ( Or maybe i really just don’t get it, since i can see a few black spots on the last image where there is no track line on at least one of the track images, so it should be also clean.)

Close enough. It calculates the most common color of all the layers. If there is two colors with the same number of frequency or if all the colors are unique, then it return nan, which explains the black dots. I will add a optional parameter to handle 2 or more colors with same frequency cases, but cases with all unique color will remain nan.

alright, should this work?

gmic sp tiger,david,fruits b 50 rep_blend_by_statistical_mode_8bit_layers

i got this:
*** Error in ./ *** inv_dim_det@[$img_checked]

OH, maybe it’s the difference in dimensions?

No, it shouldn’t work when dimensions are different. I’m not even sure how to address that, yet. So, all dimensions have to be the same for now. Also, now updated the error output.

This works rather nicely :

gmic run 'sp tiger . . r 500,500 foreach { repeat 4 { circle. {u(100)}%,{u(100)}%,{u(4)}%,{u(.7,1)},${-rgb} } } +rep_blend_by_statistical_mode_8bit_layers'

1 Like