Reptorian G'MIC Filters

@prawnsushi @grosgood Thanks, everything has been fixed, and I can continue to work on the dynamic GUI part before starting on the processing part.

A really minor news: I extended the capability of rep_cin command.

D:\Programs\G'MIC\gmic-community>gmic echo ${-rep_cin}
[gmic]./ Start G'MIC interpreter (v.3.3.5).
Hello there. My name is Reptorian. This is a test of this rep_cin command.  I'll be able to use these keys: ~!@#$%^&*()_=+-,./\][` And all seems to work fine and dandy!
[gmic]./ End G'MIC interpreter

Minor because there’s only few use case for this thing. I can see myself using this in a interactive filter where I want to save a file given a directory location. And for obscure scripts that has no place in G’MIC like the one I did entirely for fun (Brainfuck Interpreter).

Another update:

I now have working Dynamic GUI for the replacement to OOBS filter.

Gist is in here: Current Code to a filter I'm working on · GitHub

You can play with it to see how it works.

I think grosgood could try to examine how the code works, and get ideas for tutorials.

After a whole day of debugging my new algorithm for converting binary to decimal, I was able to trim my time.

D:\Programs\G'MIC\gmic-community>gmic tic rep_bin2dec 1${rep_random_bits\ 89999} toc
[gmic]./ Start G'MIC interpreter (v.3.3.6).
[gmic]./ Initialize timer.
[gmic]./ Elapsed time: 0.599 s.
[gmic]./ End G'MIC interpreter.

D:\Programs\G'MIC\gmic-community>gmic tic rep_bin2dec 1${rep_random_bits\ 89999} toc
[gmic]./ Start G'MIC interpreter (v.3.3.6).
[gmic]./ Initialize timer.
[gmic]./ Elapsed time: 1.509 s.
[gmic]./ End G'MIC interpreter

Top one is new, bottom one is old. If it was single-threaded only for new one, it’d be 1 s.

The new one is essentially the old algorithm as in the idea* still remains the same, but enables multi-threading, and more compressed code.

*idea = The usage of Schönhage-Strassen multiplication algorithm, and base 10-million.

Also, this verifies it works:

D:\Programs\G'MIC\gmic-community>gmic echo ${rep_bin2dec\ 1110100000000000000000111111111111111111111100000000000001111111111111111111111100000000000101010101010101001011111111111111101111110000000000}
[gmic]./ Start G'MIC interpreter (v.3.3.6).
5052513913269773216455623635086885525584896
[gmic]./ End G'MIC interpreter.

D:\Programs\G'MIC\gmic-community>python
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> 0b1110100000000000000000111111111111111111111100000000000001111111111111111111111100000000000101010101010101001011111111111111101111110000000000
5052513913269773216455623635086885525584896
>>> exit(

The rewrite of OOBS filter is now here. It’s available for CLI and GUI, however I still have to address the paste to CLI thing. @TasMania17

This is for G’MIC coders - Rewrite of OOBS filter · GreycLab/gmic-community@434d7ce · GitHub

Hi Reptorian, I tried the new OOBS filter. It’s a big file and I get errors unfortunately.

I use Ubuntu/Debian and the stand-alone GMIC-Qt. Because I open with a terminal I can see errors and also any applied GMIC command.

GMIC-Qt

I don’t need the ‘cli thing’ because I already have it. When I’m happy with a preview and ‘apply’ I can look at the command in the terminal. I paste that command into a batch file and run it over a thousand or more video frames. It works well.

So when I run this OOBS I see this error in my terminal:

[gmic]./fx_rep_cbsoo_preview/ *** Error *** Command ‘done’: Not associated to a ‘for’, ‘foreach’, ‘local’ or ‘repeat’ command within the same scope.
[gmic_qt]./error/ When running command ‘v 3 fx_rep_cbsoo_preview 0,1,0,0,0,“temp_text”,-1,-1,-1,-1,-1,0,1,0,0,0,1,1,0,0,1,1,0,0,0,0,3,100,1,21,100,3,100,1,21,100,1,3,100,1,21,100,1,1,3,100,1,21,100,1,1,3,100,1,21,100,1,1,3,100,1,21,100,1,3,100,1,21,100,1,0,0,0,0,2,0,1,0,2,50,50,1,1,0,“-1”,0’, this error occurred:

I think there maybe other errors. I also get an error with Vibrato that is included. BTW this a formidable work on you part.

Okay, I made final changes to allow direct pasting from GUI filter into CLI filter to work as expected after running though I have not tested it.

I also fixed up some bugs on my end that I did not noticed.

Hopefully, it’ll work this time and the update should arrive by tomorrow.

An excellent filter. It’s a lot more advanced than the earlier Channel overblur filters. Those earlier errors were probably at my end because I thought the git file was it not the G’MIC downloaded one.
It’s cool you can do that :slight_smile:

1 Like

I made a new CLI command that some coders might find useful. Thanks for David to point out to get/set, and I went to look into v2s to solve it:

It’s rep_extract_unique_values_and_count, and it doesn’t have arguments. I should add a warning that it only works on int though, but eh.

You can see output here:

D:\Programs\G'MIC\gmic-community>gmic (1,1,1,5,5,5,5,5,5,2,2,2,3,3,3,3,3,3,3,3,3,3,3,3) rep_extract_unique_values_and_count
[gmic]./ Start G'MIC interpreter (v.3.3.6).
[gmic]./ Input image at position 0, with values (1,1,1,5,5,5,5,5,5,2,2,2,3,3,3,3,3,3,3,3,3,3,3,3) (1 image 24x1x1x1).
[gmic]./ Display image [0] = '[unnamed]'.
[0] = '[unnamed]':
  size = (4,1,1,2) [32 b of float32].
  data = (1,5,2,3^3,6,3,12).
  min = 1, max = 12, mean = 4.375, std = 3.46152, coords_min = (0,0,0,0), coords_max = (3,0,0,1).
[gmic]./ End G'MIC interpreter.

1 has 3 occurrence, and it appeared first.
5 has 6 occurence, and it appeared second.

Leaving this pushed code here for example:

#@cli rep_extract_unique_values_and_count:
#@cli : Extract Unique values from a image starting from 0,0,0,0, and then outputs unique values only with the count.
rep_extract_unique_values_and_count:
foreach {
  current_img_ind=key_{$>}_

  {whds#-1},1,1,2

  eval[-2] >"begin(
    key='"$current_img_ind"';
    mi=0;
   );
   isnan(value=get([key,v2s(i,16,16)]))?(
    I[#-1,mi]=[i,1];
    set([key,v2s(i,16,16)],mi);
    ++mi;
   ):(
    ++i(#-1,value,0,0,1);
   );
   end(resize(#-1,mi,1,1,2,0););"

  rm..
}

And I like that G’MIC can support something akin to dictionary in math parser.

Anyway, this is a good case study for @prawnsushi on dictionary.

It’s not a particularly fast solution, but for parsing small images, that is just fine and dandy.

Ok, I’d like to know what the hell is going on here because it sure ain’t in the code!

It sometimes output to black, other times it works.

The code in question:

#@cli rep_lavander_binary_map: _size_of_arr_by_power_of_two>0,_mode={ -1=custom | 0=and | 1=or | 2=xor },_bin_a,_bin_b,_shift>0,custom_expression
#@cli : Create a texture that does the following steps:
#@cli : 1) Create an array of integer that matches index with size of 2^n.
#@cli : 2) Create another array which is the count of 'bin_a' in binary representation of the previous array.
#@cli : 3) Count sort the array of integer with the count of binary number array.
#@cli : 4) Delete all other array other than the sorted array.
#@cli : 5) Generate a surface with row and column the same size as the primary array, and use formula which takes values from the sorted value, and then find the count of 'bin_b' in binary representation of the found value.
#@cli : - Built-in Macros -
#@cli : flip(v) - Flips the bitwise values of number
#@cli : and(a,b) - a & b
#@cli : or(a,b) - a | b
#@cli : - End of Built-in Macros -
#@cli : Note : The idea comes from Lavander at discord/generative. Based on a Python code.
#@cli : Default values: '_size_of_arr_by_power_of_two=11','_mode=2','_bin_a=01','_bin_b','_shift=0',
+new_rep_lavander_binary_map:
skip ${1=11},${2=2},${3=01},${4=10},${5=0},${6=}

check "$1>0&&($1==int($1))&&(isint($2)||same('-','$2'))"

a6={size('$6')}
shift,length,mode:=int(abs($5)),1<<$1,v=same('-','$2');n=isint($2)?$2:0;v?0:(n>=0?n%3+1:-((abs(n)-1)%3+1))
size_of_arr={int(log2($length+$shift-1))+1}
str_code=n=init_num()
act_arr_sortclass={$mode>-1}

if !$mode&&!$a6 error custom_formula_required fi

$length,1,1,{1+$act_arr_sortclass},>"begin(
         const first_binary_to_search=0b$3;
         const second_binary_to_search=0b$4;
         const size_of_first_binary_to_search=size('$3');
         const size_of_second_binary_to_search=size('$4');
        );
        count=0;
        current_value=x;
        current_number_of_binary_digits_per_value=(i?int(log2(current_value)))+1;
        bit_mask=(1<<current_number_of_binary_digits_per_value)-1;
        c?(
         binary_to_search=second_binary_to_search;
         size_of_binary_to_search=size_of_second_binary_to_search;
        ):(
         binary_to_search=first_binary_to_search;
         size_of_binary_to_search=size_of_first_binary_to_search;
        );
        while(current_number_of_binary_digits_per_value>=size_of_binary_to_search,
         test_section=current_value>>(current_number_of_binary_digits_per_value-size_of_binary_to_search);
         test_section==binary_to_search?(
          ++count;
          bit_mask>>=size_of_binary_to_search;
          current_number_of_binary_digits_per_value-=size_of_binary_to_search;
         ):(
          bit_mask>>=1;
          --current_number_of_binary_digits_per_value;
         );
         current_value&=bit_mask;
        );
        count;
        "
 
if $act_arr_sortclass
 
 $length,1,1,1,x rv[-2,-1]
 
 +channels. 0 # Create copy as we will preserve the the generated output as lookup table

 if $shift -. {im} fi

 # Pixel sort integer by count sort
 +histogram. 100%
 f[-1] >begin(n=0;);i?(v=n;n+=i;v;);
 f[-2] >v=i(#-1,i);++i(#-1,i);v;
 {w#-2},1,1,1
 eval[-3] i(#-1,i)=i(#-5,x);
 rm[-5,-3,-2]
 a[-2,-1] c
 
fi

if !$mode str_code..=ix=max_index-x;iy=max_index-y; fi

binary_set_to_search,size_of_binary:=$act_arr_sortclass?[size('$4'),0b$4]:[size('$3'),0b$3]

$length,$length,1,1,:"begin(
  const shift_factor=$shift;
  const mode=$mode;
  const binary_set_to_search=$binary_set_to_search;
  const size_of_binary_set=$size_of_binary;
  const reference_channel=1+$act_arr_sortclass;
  const lookup_table_pos=$act_arr_sortclass;
  flip(v)=xor((1<<(int(log2(v))+1))-1,v);
  and(a,b)=a&b;
  or(a,b)=a|b;
  x()=i(#-1,x,0,0,reference_channel,2);
  y()=i(#-1,y,0,0,reference_channel,2);
  ix()=i(#-1,max_index-x,0,0,reference_channel,2);
  iy()=i(#-1,max_index-y,0,0,reference_channel,2);
  mode==3?(
   init_num()=xor(i(#-1,x,0,0,reference_channel),i(#-1,y,0,0,reference_channel));
  ):
  mode==2?(
   init_num()=i(#-1,x,0,0,reference_channel)|i(#-1,y,0,0,reference_channel);
  ):
  mode==1?(
   init_num()=i(#-1,x,0,0,reference_channel)&i(#-1,y,0,0,reference_channel);
  ):
  mode==-1?(
   shift_factor?(
    init_num()=(x+shift_factor)&(y+shift_factor);
   ):(
    init_num()=x&y;
   );
  ):
  mode==-2?(
   shift_factor?(
    init_num()=(x+shift_factor)|(y+shift_factor);
   ):(
    init_num()=x|y;
   );
  ):
  mode==-3?(
   shift_factor?(
    init_num()=xor(x+shift_factor,y+shift_factor);
   ):(
    init_num()=xor(x,y);
   );
  ):(
   init_num()=int("$6");
  );
 );
 count=0;
 "$str_code";
  count=0;
  n=abs(n);
  !inrange(n,0,w,1,0)?(
   num_of_binary_digits_of_n=(n?int(log2(n)))+1;
   bit_mask=(1<<num_of_binary_digits_of_n)-1;
   while(num_of_binary_digits_of_n>=size_of_binary_set,
    test_section=n>>(num_of_binary_digits_of_n-size_of_binary_set);
    test_section==binary_set_to_search?(
     ++count;
     bit_mask>>=size_of_binary_set;
     num_of_binary_digits_of_n-=size_of_binary_set;
    ):(
     bit_mask>>=1;
     --num_of_binary_digits_of_n;
    );
    n&=bit_mask;
   );
   count;
  ):(i(#-1,n,0,0,lookup_table_pos));"

rm..

if !$act_arr_sortclass&&$shift -. {im} fi

Even worse, the GUI filter is all messed up with this new code. Just remove new_ to find that out!

EDIT:

Seems that it has to do with the image before $length,$length,1,1 part. I will investigate it, but it is a weird issue. Ok, I can trace it down to the $length,1,1,{1+$act_arr_sortclass}, but I see no reason why this should happen. So, I don’t know why, so there is something really wrong here.

1 Like

I don’t have this problem on Linux.
Maybe it’s just display playing tricks on you? Just a shot in the dark, but have you tried moving the image around, or zooming, when it’s black? Or even resizing the window? :shushing_face:
It’s also almost pitch black when i turn off normalization.

Forget it, i see you have only zero’d pixels in your video.
Still, i can’t get it to fail.

Maybe it’s a OS-specific thing. I traced it down to a single variable which is current_number_of_binary_digits_per_value.

On this block: $length,1,1,{1+$act_arr_sortclass} : Add this line of code.

if(x==128,print(current_number_of_binary_digits_per_value));

My results:

C:\Windows\System32>gmic +new_rep_lavander_binary_map 8,0
[gmic]./ Start G'MIC interpreter (v.3.3.6).
[gmic_math_parser] current_number_of_binary_digits_per_value = (uninitialized) (mem[45]: scalar)
[gmic_math_parser] current_number_of_binary_digits_per_value = 8
[gmic_math_parser] current_number_of_binary_digits_per_value = 8
4::MSB1aW50OCBsaXR0bGVfZW5kaWFuCjI1NiAxIDEgMiAjODIKeJydTkkOACEIg9b/v3mmYg96IFET04WyRPwv9TONCfFdJwD7XV0Iiu/5135rkLTfzb/eH4vOUKGip64jyu/qQp165l/7rclB+9382/0fpZ4DAg==
[gmic]./ Display image [0] = '[begin( const shift_factor=$s...'.
[0] = '[begin( const shift_factor=$shift; const mode=$mode; const bi...':
  size = (256,256,1,1) [256 Kio of float32].
  data = (0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, ... ,1,1,1,2,2,2,2,3,3,3,2,2,2,3,1,1,1,2,2,2,1,2,2,2,2,2,2,3,3,3,2,2,2,3,2,1,1,1,2,2,2,2,3,3,3,2,2,2,3,2,1,1,1,2,3,3,3,2,1,1,1,2,3,3).
  min = 0, max = 4, mean = 1.3125, std = 0.802833, coords_min = (0,0,0,0), coords_max = (251,35,0,0).
[gmic]./ End G'MIC interpreter.

C:\Windows\System32>gmic +new_rep_lavander_binary_map 8,0
[gmic]./ Start G'MIC interpreter (v.3.3.6).
[gmic_math_parser] current_number_of_binary_digits_per_value = (uninitialized) (mem[45]: scalar)
[gmic_math_parser] current_number_of_binary_digits_per_value = 1
[gmic_math_parser] current_number_of_binary_digits_per_value = 1
0::MSB1aW50OCBsaXR0bGVfZW5kaWFuCjI1NiAxIDEgMiAjMTQKeJxjYBgFIxkAAAIAAAE=
[gmic]./ Display image [0] = '[begin( const shift_factor=$s...'.
[0] = '[begin( const shift_factor=$shift; const mode=$mode; const bi...':
  size = (256,256,1,1) [256 Kio of float32].
  data = (0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, ... ,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0).
  min = 0, max = 0, mean = 0, std = 0, coords_min = (0,0,0,0), coords_max = (0,0,0,0).

@David_Tschumperle Any explanations on this perplexing bug? It’s perplexing because:

        current_value=x;
        current_number_of_binary_digits_per_value=(i?int(log2(current_value)))+1;

When x is 128, it should be 8. Not 1. So, there should be no reason why I get a 1 here randomly. It is usually 8 though. With multiple computers, this bug does show up on Windows. That’s for certain.

Could you please write a minimal code to reproduce the issue ?
Unfortunately, I can’t spend hours understanding a large chunk of code to find a possible bug.
Thanks!

I fixed it by separating x part and using i on fill instead. I"ll try to find a minimal example, but hopefully I can find that because this is the only time I have ever seen this issue.

After which line exactly?
Placed it under this:
current_number_of_binary_digits_per_value=(i?int(log2(current_value)))+1;
But it doesn’t print anything.

Ok, leave it to me to find the minimal example. I pushed the new Lavander Binary Map after I fixed bugs.

Old one with using 11 (2048x2048) : 0.667 s
New one with using 11 (2048x2048) : 0.057 s

~12x faster! It’s using lookup table now. That’s all the practical changes.

I’m just gonna save this here and bookmark it myself - https://cs.wellesley.edu/~pmetaxas/ei99.pdf

Apparently, it’s possible to parallelize dithering effect.

Now, I’m releasing a new version of Color Harmonies for testing:

So, if you’re able to test it, import this via G’MIC plugin, and then let me know how it worked out.

Also, about the earlier message, I wanted to know if value() is just invisible text(). And isn’t there an alternative name because value() seems to be giving the impression it’s just number.

To which post are you referring?

Recently deleted post.

Here on Linux, it seems to work as expected.
Not sure what changed from the old version (besides the points size, and the coords box missing), but it works :wink:

Also tried the CLI version :

Looks pretty useful for color amputees like me :slight_smile: