Making the bridge between math expressions and the G'MIC interpreter


(David Tschumperlé) #1

Note: This is a quite technical post about new features coming in next G’MIC version 2.0.4. It probably won’t get any interest from most users, sorry about that! :slight_smile:

If you already had a play with the math expression parser in G’MIC (as @grosgood did here), you may have noticed that the math parser defines its own mini-language by itself inside the G’MIC interpreter. Actually, the possibilities brought by this little thing are really interesting. Speaking for myself, as I’m adding new commands in the G’MIC stdlib, I’m using the math expression evaluator more and more.
And sometimes, this is absolutely not for doing ‘usual’ calculus.

For instance, I recently had to write a G’MIC command that check if all the lines of a .cpp/.h source file have less than 120 characters. My first attempt was something like this :

check120 : 
  -i raw:"$1",uchar
  -s +,{'\n'}
  -repeat $!
    -if {$>,"i!=_'\n' && h>120"} -error "One line has more than 120 chars !" -endif
  -done
  -rm

which actually works fine. But when the file you want to inspect is large (more than 60.000 kloc for me), then the command -check120 takes a lot of time to achieve. Because the -repeat...-done loop is interpreted and when the number of iterations is large, this is slow.
On the contrary, the G’MIC math parser embeds a JIT compiler that first compiles your math expression into bytecodes, then interpret the bytecode sequence. This is still interpreted, but as it is already pre-compiled into bytecodes, the evaluation is really faster than the loop written as a G’MIC pipeline.
So my command -check120 became:

check120 :
  -i raw:"$1",uchar
  -s +,{'\n'}
  -skip {"
    for (k = 0, k<l, ++k,
      if (i[#k,0]!=_'\n' && h#k>120, print(One_line_has_more_than_120_chars=0));
    );
  "}
  -rm

And now it’s instant. I won’t enter in the details here and there, but that’s one reason why - as a G’MIC stdlib developer :wink: - I use the G’MIC math expression evaluator more and more, and as a result, the expressions I’m writing become bigger and more complex as the time goes by.

So, I reached a point where occasionally, I’d like to implement a command in a reverse way, which means that instead of writing a G’MIC pipeline that uses one or two calls to the math parser, I’d like to write a command as a single math expression that could be able to invoke one or two G’MIC pipelines.
And that is precisely what I’ve implemented this week end, so let my show you a little bit how this is intended to work in the next G’MIC version (you can also follow these changes in real time here).

Now, the math parser defines a new function ext() that is able to invoke a G’MIC pipeline inside a math expression.
So, something like this will be possible:

foo : 
  -sp rose,tiger
  -eval "
    for (i = 0, i<10, ++i,
       ext('-blur[0] 1');
   );"

which applies a blur on the image [0], 10 times.
Two points to note:

  • The argument of the ext() function is a vector representing a string (contains the ascii codes of the string characters).
  • The ext() function invokes the same G’MIC instance than the current command -foo, so it shares the list of images and image names, as well as the local and global variables.

Of course, you can imagine that having the possibility to build custom strings for ext() is mandatory, so I’ve added a few functions to convert values/vectors to strings. If you pass multiple string arguments to ext(), then all the strings are concatenated to form the final G’MIC pipeline to invoke inside the math parser.
For instance, this command:

foo : 
  -sp rose,tiger
  -eval "
    for (i = 0, i<10, ++i,
       ext('-blur[',dtos(round(u)),'] ',dtos(u*10));
   );"

will try to blur 10 times, either the image [0] or the image [1] with a random std between 0 and 10.
(function dtos() stands for ‘Double TO String’).

You get the idea: ext() allows you to invoke a G’MIC command line inside a math expression, with a command line that can be generated on the fly -> so many possibilities ! :wink:

I haven’t thought of all possibilities this new ext() function brings, but I can already tell that it will be useful to set a G’MIC variable inside a math expression, like in the example below:

foo :
  -sp lena
  count100=0 count200=0
  -f "if (i==100,
        ext('count100+=1'),
      if (i==200,
        ext('count200+=1');
      )); i"
  -e $count100,$count200

Here, I’m counting the number of occurences of pixel values 100 and 200 and updates the two different G’MIC variables directly during the math expression evaluation.
We can also imagine now that a math expression can insert or remove images in the list, or rearrange the list order, etc…

Well, it’s late right now so I won’t tell more about that, but if you are curious enough, please read the evolving changelog and start to play with these features (you just have to compile G’MIC from the current git master). Any feedback will be appreciated !


#2

Beyond my capabilities to understand. But, your enthusiasm decorates my monitor now. Thank you for your years of work, you share with us.


(David Tschumperlé) #3

I’ve just posted new pre-release binaries for next version 2.0.4 today on the website, including the plug-in binaries for GIMP. Feel free to test and report possible regressions ! Thanks :slight_smile:


#4

Hah! I think G’MIC just turned inside-out :crazy_face:
It makes me wonder if the future might be to run JIT by default and embed the rest of G’MIC inside it…

Hmm I wonder what happens if I call -eval within an extern() block! Or -command or -foo. I think my head’s about to explode :slight_smile:


(David Tschumperlé) #5

Well, that is indeed possible that in the future, some commands may be only composed of a single -eval invokation, with some ext() inside. But this won’t be probably useful for all commands.

It’s not that weird actually. Just think of it as an interlude in the evaluation of the math expression, where you are allowed to run some G’MIC pipeline. For instance, this command does work as expected :

foo :
  -eval "
    ext('-eval \"pi\" -echo ${}');
  "

gives this:

$ gmic -foo
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./foo/ Evaluate math expression ' -ext('-eval "pi" -echo ${}'); '.
[gmic]-0./foo/*ext/ Evaluate math expression 'pi'.
[gmic]-0./foo/*ext/ 3.1415926535897931
[gmic]-0./ End G'MIC interpreter.

This thing will be awesome :smiley:


(David Tschumperlé) #6

Edited posts to replace extern() by ext() :wink: