New useful features in math parser, planned in 2.9.8

Software G'MIC
1

I wanted to make a specific post about this, as I know there are heavy/advanced users of the math parser here, including @Reptorian, @grosgood and @afre among others.

This morning, I’ve just added two new functions to the G’MIC math parser, and I believe they will be quite useful in practice.
Both are quite similar and have two valid signatures:

  • fill(target,loop_varname,expression) and fill(target,loop_varname,expression) fill the object target (useful mainly for vector-valued objects) with the given expression, with eventually specifying a name for the index counter.

For instance:

eval "
  V = vector8();
  fill(V,k,k<4?k^2:k-4);
  print(V);
  fill(V,round(u(100)));
  print(V)"

which gives:

[gmic]-0./ Start G'MIC interpreter.
[gmic_math_parser] V = [ 0,1,4,9,0,1,2,3 ] (size: 8)
[gmic_math_parser] V = [ 40,41,15,23,14,26,81,30 ] (size: 8)
[gmic]-0./ End G'MIC interpreter.
  • And the other function, repeat(), works almost the same, it is just equivalent to a for (k = 0, k<nb_iter, ++k) (but more faster and shorter to write). It has also two possible signatures:
eval "
  repeat (10,k,print(k));  # eq. to 'for (k = 0, k<10, ++k, print(k));'
  repeat (10,print(u)); # eq. to 'for (k = 0, k<10, ++k, print(u));'
"

The second variant of these function is useful when no counter variable is actually needed.

Note that when specifying a variable, it is *not local to the command call, meaning that:

eval "
  repeat (10,foo,print(foo));
  echo('foo is now ',foo);
"

gives:

gmic]-0./ Start G'MIC interpreter.
[gmic_math_parser] foo = 0
[gmic_math_parser] foo = 1
[gmic_math_parser] foo = 2
[gmic_math_parser] foo = 3
[gmic_math_parser] foo = 4
[gmic_math_parser] foo = 5
[gmic_math_parser] foo = 6
[gmic_math_parser] foo = 7
[gmic_math_parser] foo = 8
[gmic_math_parser] foo = 9
foo is now 9
[gmic]-0./ End G'MIC interpreter.

Also functions break() and continue() can be used in the fill() and repeat() functions to stop at a given iteration, or continue to the next iteration (in the case of fill(), using continue() won’t set the value V[k] for the kth iteration).

Well, that’s it for now. I’ll probably use those functions a lot in the future.
I’m currently trying to implement a neural-network library inside G’MIC, and these functions will help, for sure.

1 Like
2

Basically, expr(), but with the option of allowing one to use variable inside the math parser. I must do some tests here. I think I’ll use it in the future, I needed this before.

3

I did some more modifications, and now it seems it is working as expected.
Timing are good, a use of repeat() is roughly 30% faster than the equivalent for() invokation.
Right now, I’m building pre-release packages for 2.9.8 with these changes, hopefully available in one hour or so,for testing (not all flavors of OSes, but mainly Ubuntu and Windows).

4

New pre-release binaires are ready!

5

I haven’t really seen much of a performance change when testing with rep_tfrac, but it is an optimization I will use, I’ll replace the for loop in my gmic file with repeat. 7% faster on my end.

EDIT: Done replacing them.

Also, now I have a question for @David_Tschumperle. What if I want offset in repeat()? I noticed that when replacing some for loop to replace, there were some that I didn’t want to replace because they start with 1 instead of 0 or some other numbers.

Another thing, it did made Attractor filter way faster.

6

When I said a 30% gain, I was only comparing the time spent to manage the loop, so not timing the interior of the loop itself.
I may add an option later for repeat() and fill() to possibly enable multi-threading.

If you have an offset in your loop index, or a non-unitary counter step, then keep using for(), it is probably better.

7

On repeat, negative number/positive number can be used to determine if it to be multithreaded. That’s just to keep in 2 signatures convention. Or, is it gonna be with * , : ?

8

Actually, managing multi-threaded loops in the math parser is really more complicated than I expected, because this basically means each thread must have its own loop counter, but at the same time, the expression compiled by the JIT as the third argument of fill() can only generate a bytecode with a single memory slot for the loop counter.

For this reason, I think I’ll just make it single-threaded.

1 Like
9

Some news :

Yesterday, I’ve worked on optimizing the math expression parser in CImg/G’MIC. It’s a bit technical:
I’ve basically improved the bytecode generation by the JIT compiler in order to evaluate the math expression, by limiting the number of copies of chunks of memory. This happens when initializing a variable, like in

X = vector512();

Before, a temporary vector512() object was first created in memory, then copied into another vector512() object assigned to the variable X. Which means, two vector512() objects were created for this single line. But if the variable X was not defined before, it was not optimal, because we could just create a single vector512() and assigned its memory slot to X. This is what is done now.

Before, as a work-around, I had added function ref(vector512(),X) to explicitely says to the JIT compiler that the variable X must be assigned to the vector512() created. This is not necessarily anymore, as the compiler is “smart enough” to notice that when X is initialized the first time, there is no need to copy the right-hand operand of the = operator to X.

As a result, the evaluation of math expressions should take less memory than before.

This works actually in a lot of situations, like:

A = expr('x + y',10,10);
E = eig(A);  # <- This won't require a copy anymore, with 2.9.8

This was not so easy to achieve, but it looks like it works now. I’ve updated the binaries of the 2.9.8_pre version on the G’MIC website for all OSes, so do not hesitate to test and report strange behaviors.

10

Some news (2021/05/17):

I’ve added a new operator $ in the G’MIC math parser, allowing to retrieve the value of an ‘external’ variable (i.e. a variable defined in the G’MIC interpreter) as a const number in the math parser.
An example is better than 1000 words:

foo : 
  var=1976
  eval "
    const best_year = $var;   # <- No need to break the double-quotes anymore!
    print(best_year)"

Note that before, it would be required to write something as:

foo_old : 
  var=1976
  eval "
    const best_year = "$var";
    print(best_year)"

So, this seems to add only little value.
But it’s a little bit more subtle than just removing the need for double-quotes:

With the new syntax, the variable is retrieved at the compilation step, so it can be used in macros for instance:

foo :
  sp david,colorful,bottles,earth
  eval "
    print_pos(name) = (
      unref(ind,W,H,D,S);
      const ind = $name#;
      const W = w#ind;
      const H = h#ind;
      const D = d#ind;
      const S = s#ind;
      echo('Position of image \'name#\' is [',ind,'] (image ',W,'x',H,'x',D,'x',S,').');
    );
  print_pos(earth);
  print_pos(colorful);
  print_pos(david);
  print_pos(bottles)"

gives:

$ gmic foo
[gmic]-0./ Start G'MIC interpreter.
Position of image 'earth' is [3] (image 500x500x1x3).
Position of image 'colorful' is [1] (image 800x800x1x3).
Position of image 'david' is [0] (image 800x600x1x3).
Position of image 'bottles' is [2] (image 750x500x1x3).

Note that the evaluation of $varname is done during the compilation of the math expression, not during its evaluation. So if your expression changes the value of $varname (for instance, by using run('varname=something');), then $varname will keep its initial value read during the compilation (it’s a const value).
To evaluate a variable during code execution, use get() instead.

1 Like
11

Helpful changes. Makes it easier for me to navigate the math parser (I will still bother you all about it… :blush:).

Does it mean that ref() is no longer necessary because it is done auto-magically?

What does the # mean here?

const ind = $name#;

I found the reason for #!

Number signs appearing between macro arguments function actually count for empty separators. They may be used to force the substitution of macro arguments in unusual places, e.g. as in

str(N) = ['I like N#'];

1 Like
12

Yes, actually the need for ref() will be greatly reduced for the good sake.
ref() still has use cases anyway, but mainly to write macros now.

13

Is it faster to write macros within ref()? Usually, what I do is “switch-case” or if…elseif…else block and assign a macro since it’s so much more neater-looking.

14

Not sure what you mean by this. Could you explain more what you have in mind?

15

To me, macro means something like this within math parser:

  $3==5?(
   fnx()=x+rot_x(j,k);
   fny()=y+rot_y_alt(j,k);
  ):
  $3==4?(
   fnx()=x+rot_x(j,k);
   fny()=y+rot_y(j,k);
  ):
  $3==3?(
   fnx()=x+rot_x(j,0);
   fny()=y+rot_y(j,0);
  ):
  $3==2?(
   fnx()=x;
   fny()=y+k;
  ):
  $3==1?(
   fnx()=x+j;
   fny()=y;
  ):(
   fnx()=x+j;
   fny()=y+k;
  );

So, I wanted to know if ref() would make this faster.

16

No, ref() has to do with variables, not macros.
Actually, you should avoid defining macros as you did, because it is prone to errors.

Macros are determined at compile time, and what you wrote works in this case, because the different conditions can be determined at compile time also (they involve only constant values).
But writing this is a bit dangerous, because if now, you add a condition that cannot be determined at compile time (like u<0.5), then the macro will be defined with the latest definition found, which will be probably hard to debug.

A macro should always be defined once, and you can put you ‘constant conditionals’ in it with no cost at all:

fnx() = (
  $3==5?(
     ...
  ):( 
     ...
  );
);

This is a general rule to know : as long as you deal with constant values, the JIT compiler will try to avoid compiling stuffs that cannot happen during the evaluation phase. So at the end, your fnx() macro will contain only the minimal code that is necessary to evaluate your expression.

17

On the $ function within interpretator, I’m not seeing that it works on fill “”.

[gmic] *** Error in ./rep_random_gradient_bars/*repeat/*local/ *** Command 'input': Unrecognized item '$sub' in expression '...const bar_width=25*$sub...'.
18

Can confirm. It worked before; it doesn’t now.

19

Have you tried with the very latest dev version ?

20

Yes. Reinstalled and updated: still a problem.