G'MIC math evaluator now deals with vectors (and matrices, and custom functions!)

(G'MIC staff) #1

Hello there,

I’ve written a lot of code these last days on a specific feature of the G’MIC framework, namely the math expression evaluator, and I guess it’s time for me to write a first report about the advances I’ve made.
This is still work in progress, so any comments are welcome to improve it before the release of version 1.6.9.
All this has been mainly motivated by this post by Martin (aka Souphead), a Gimpchat member. Basically, he wanted the math evaluator of G’MIC to be able to deal with complex numbers rather than only scalar numbers. So that’s what I’ve tried to improve :slightly_smiling:
The cool thing is, I didn’t break anything, just added some new stuff to the math evaluator while keeping a full compatibility with the previous syntax. To get an idea of the work I’ve done, this required me to add approx 3500 lines of C++ code to the math parser (that has now 5000 loc instead of 1500). So, this was not really a cakewalk!

So, a G’MIC math expression is now able to manage two different kinds of ‘objects’ : scalars and vectors.
A vector can have an arbitrary size, but its size must be known at ‘compile time’ (when the math expression is transformed into a serie of bytecodes before being evaluated). Basically, the vector dimension is a fixed constant and cannot be modified once a vector variable with that size has been declared. Of course, you can declare several vector variables having different dimensions.

A new vector can be constructed (and optionally assigned to a variable) using either the new operator [...], or the new function vector():

A = [ 1,2,3,4 ];  # Construct a 4-dimensional vector, with values (1,2,3,4)

or

B = vector4(1,2,3,4);   # Does the same as 'A'
C = vector(1,2,3,4);    # Does the same as 'A'

The vector() function can be useful to construct large vectors, without having to type all the coefficients, like in this example:

A = vector256(0);  # A 256-dimensional vector filled with '0'

Another example:

vector8(0,1,2);  # This corresponds to the vector '[ 0,1,2,0,1,2,0,1 ]'

Well, as you see, that is quite easy. Now, you have to know that a math expression in G’MIC can thus return either a scalar, or a vector:

  • When a vector is returned in a substituting expression as {formula}, the expression substituted by the string composed of the vector values, separated by commas. For instance:

      $ gmic image.jpg -echo Dimensions={";[w,h,d,s]"}
    
      [gmic]-0./ Start G'MIC interpreter.
      [gmic]-0./ Input file 'image.jpg' at position 0 (1 image 512x512x1x3).
      [gmic]-1./ Dimensions=512,512,1,3
    

(I’ve put a ; as the first character of the math expression here to make clear the substituting expression is a math formula rather than another possible substituting expression, see doc for more info).

  • And if you use a vector-valued math expression in the -fill or -input commands, then it actually fills each of your image pixel directly with the corresponding vector. For instance,

      $ gmic 256,256,1,3,'[x,y,(x+y)/2]'
    

generates this 256x256 color image, with red=x,green=y and blue=(x+y)/2:

Note that it was of course possible to do the same before, but with a more ‘cryptic’ expression (scalar-valued), for instance:

$ gmic 256,256,1,3,'!c?x:c==1?y:(x+y)/2'

This principle works also for all commands accepting math expressions to modify image values (so all arithmetic and boolean operators, etc…).

Note also that in an expression, you can access the pixel of an image directly as a vector value with expressions I or I(x,y,z). Actually, all previous expressions that was defined to allow pixel access (as scalar values) have been extended to their vector-valued counterparts. You just have to use capital letters I and J rather than i and j to make them ‘vector-valued’.

Also, all operators and functions of the math evaluator have been extended as well to deal with vector-valued parameters. For instance, expression abs([ -1,-3,2,-4,5 ]) returns the vector [ 1,3,2,4,5 ].

Managing complex numbers : So, now, a complex number can be defined simply as a 2-dimensional vector [ real,imaginary ]. I’ve added some new functions and operators to deal specifically with complex numbers, that is the operators ** (complex multiplication), // (complex division), ^^ (complex exponentiation), **= (self-complex multiplication), //= (self-complex division), ^^= (self-complex exponentiation), and the functions cabs() (complex modulus), carg() (complex argument), cexp() (complex exponential), clog() (complex logarithm), cconj() (complex conjugate).
With these tools, writing the rendering of a Mandelbrot fractal becomes very simple. That’s the function you can try:

mandelbrot:
  1024,1024,1,1,"
z = 2.4*[ x/w, y/h ] - 1.2;
for (iter = 0, cabs(z)<=2 && iter<256, ++iter, z = z**z + [ 0.4,0.2 ]);
iter
"
  -n. 0,255 -map. 7

Other new functions: Some new functions have been also introduced specifically to deal with vector-valued arguments : sort(A,is_increasing) (to sort the values of a vector A in increasing or decreasing order), cross() to compute the cross product between two 3d-vectors, size(A) which returns the number of values in vector A.

Some functions signatures have been also extended to accept vector-valued arguments, for instance you can access pixel values using vector-valued coordinates:

X = [ x,y,z ]; pixel_value = i(X);

or you can find the minimum value in a vector, like this

X = [ 3,5,7,1,1,2,8,3 ]; min_value = min(X);

The operator [] also allows to access a specific vector value now (a scalar):

X = [ 1,2,0,4 ]; zero = X[2];

(indexes starts from 0, as in C++).

To import/export vector-valued variables from/to images, you can use the assignment operator, like this:

V = I[#1,0];  # Get first vector value of image [1].
V = sort(V);  # Sort vector values.
I[#1,0] = V;  # Put vector result back in image [1].

Well, that’s it for now. If you have ideas and suggestions about new functions I could add to the math evaluator, that would be great! Also I want to say that doing all this required a major rewrite of some of the math evaluator code. I took the opportunity to clean and optimize the code a lot. The added complexity makes the compiler a bit slower to compile the bytecode for the expression, but it generates a shorter bytecode, with more optimized strategies and the evaluation of the expression itself is finally faster than in the previous version.

Any comments ?

EDIT: I’ve been able to upload new pre-releases in the G’MIC web site, so you can even install the development version with the improved math parser, and play with it. Check this page.

2 Likes
(Karsten R) #2

(I send it by mail, still the list is better!)

Hi David,

nice and helpful text, and large progress for the gmic scripting!

Just some remarks to your text:

IMHO: vectors are by definition 1-dimensional still with size or length! Hence I won’t call that dim()!

Your example gmic image.jpg -echo Dimensions={;[w,h,d,s]} under bash should be “gmic image.jpg -e Dimensions={;[w,h,d,s]}” at least on my machine.
[gmic]-0./ Start G’MIC interpreter.
[gmic]-0./ Input file ‘image.jpg’ at position 0 (1 image 425x329x1x3).
[gmic]-1./ Dimensions=425,329,1,3

Or “gmic image.jpg -e Dimensions={vector(w,h,d,s)}” is an example too.

Does that mean, that a vector in -fill and -input always fills the s-channel, channel 3, in its defined size/length? No change possible?

Congratulations
Karsten

(G'MIC staff) #3

Ah maybe dim() is not the best name ? Matlab uses size() or length() apparently. size() looks good to me.

No, it the returned vector size is less than the number of channels, the remaining values are unchanged.

(Karsten R) #4

That I understood, my question was more in direction of assinging in z,y or x channel, so to say not one pixel(-vector) but a vector(-pixel) skalar!

(G'MIC staff) #5

I’m not sure I understand what you really want to do.
But in any case, I think the answer is “you can do it” :slight_smile: I don’t see a case where you would have a problem
(but tell me if you think about one).

#6

Crikey! 3500 lines of your code is probably a life’s work of mine :smiley:

Sorry to be the one to ask but is this likely to slow down math parser in tight loops to any noticeable degree (without even using the new vectors)?

(G'MIC staff) #7

This shouldn’t be really noticeable, because it just adds some conditional tests in the code, nothing more.
The use of vector variables is a bit slower than using scalars though, but if you don’t use them at all, you shouldn’t even remark the compiler has been modified :slight_smile: (it should be even a bit faster than before, due to some optimizations I’ve added).

#8

Ah this is good news :slightly_smiling:
I’m sure this will bring along interesting matlab conversions and some new ideas too!

(Lyle Kroll) #9

All I have to say is more fantastic news and the preset for Droste is awesome. Thanks Martin and you too David. Much appreciated. :slight_smile:

(Lyle Kroll) #10

Lots of cool Matlab code out there. From packed circles to some cool polar plots like link below. :slight_smile:

http://www.mathworks.com/help/matlab/creating_plots/record-animation-for-playback.html?s_tid=gn_loc_drop

(G'MIC staff) #11

Episode II: Matrices are now in the game!

While thinking about the new possibilities introduced by the vector types in the math evaluator, it came to my mind that of course I’d like to be able to do some (basic) matrix operations, like multiplication, inverse, retrieving eigenvalues and so on… The CImg Library has already some interesting functions to deal with matrices (and you can use them already in G’MIC through the usual pipeline commands, see here). Why not just being able to call them inside the math parser ?
That’s what I’ve done mainly today.

I didn’t add a new matrix type to the math evaluator, because this would have make the compilation step a lot more complex, and I’m afraid this would have been not adapted for a Just-In-Time compiler as we have in G’MIC. A matrix is seen as a simple vector where all matrix coefficients have been ‘unrolled’. For instance, matrix [ 1,2,3; 4,5,6; 7,8,9 ] is represented by the vector:

A=[1,2,3,4,5,6,7,8,9];

You just have to keep in mind you want A to be a 3x3 matrix instead of a 9-d vector.
Now, suppose you want to multiply the matrix A with a 3-d vector X. Then, the operator** does that automatically.

X = [ 1,2,3 ];
B = A**X;

and you get B = [14,32,50]. In this case, you don’t have to specify the size of the matrix anywhere, as the only possible dimensions to allow a product between the matrix A (9 values) and the vector X (3 values) is when A is 3x3 and X is a 3d-vector. Note also that the operator** was already used to multiply two complex numbers together. So what happens if I write A**X with A and B being two 2d-vectors ? How can I specify I want a matrix multiplication instead of a complex multiplication? In fact, this is the only ambiguous case that can happen, and you cannot tell the operator you want a matrix multiplication (so, the complex multiplication is used here). Why that ? Because that would correspond to the multiplication of a 1x2 matrix by a 2x1 vector, which results in nothing more than the dot product between A and X, which can be easily computed by dot(A,X). So it’s not a big deal to disable this matrix product in this particular case.

So, what can of other things we can do now with these kind of matrix representations ?
A lot of things, we the newly introduced functions:

  • mdet(A): return the determinant of the (square) matrix A.
  • mdiag(V): return a square diagonal matrix from the values of the given vector V. For instance mdiag([1,2,3]) return the 3x3 matrix stored as the 9-d vector [1,0,0,0,2,0,0,0,3].
  • meig(A): return the eigenvalues and eigenvectors of the (square) matrix A. The first row of the result stores the eigenvalues in decreasing order, while the remaining rows store the corresponding eigenvectors.
  • meye(n) : return the identity matrix of size nxn (stored as a n^2-d vector).
  • minv(A): return the invert of the (square) matrix A.
  • mmul(A,B,nb_colB) : multiply two matrices A and B with arbitrary size. You just have to specify the number of columns of B to disambiguate the operation.
  • mrot(x,y,z,angle): return the 3x3 rotation matrix with specified axis (x,y,z) and angle.
  • msolve(A,B,nb_colB): return the least-square solution X to the linear system A.X = B, A and B being two arbitrary matrices. Here again, you only have to specify the number of column of B to disambiguate the function with the matrices stored as vectors.
  • mtrace(A): return the trace of the (square) matrix A.
  • mtrans(A,nb_colsA): return the transpose matrix of an arbitrary matrix A.

With all these new functions operating on matrices, we have a whole new set of possibilities for writing math expressions. G’MIC is not yet Scilab of course :), but I think it will be really useful to write more complex custom commands in the future.

What I did also today is improve and normalize the various error messages you can get when you write incorrect math expressions.
That’s it for episode II. I think I need to find an elegant way now to be able to retrieve ‘sub-vectors’ from an existing vector, and I’ll be almost done for what I want to do on the math evaluator.

Stay tuned !

EDIT: I found out finally how to create a sub-vector access. I think I’ve done everything I was thinking of.
Time to sleep now, until you have some nice suggestions to implement :slightly_smiling:
So, you can create a sub-vector, using the array brackets [] , like this:

X = [ 1,2,3,4 ];
Y = X[0,2];    # Return vector [ 1,2,3 ]

And as you can also append vectors together, you can of course get some pieces at different places from a vector:

X = [ 0,1,2,3,4,5,6,7,8,9 ];
Y = [ X[0,2], X[4], X[8,9] ];   # Return vector [0,2,4,8,9];

Feel free to comment !

(G'MIC staff) #12

Episode III : Custom functions

A new feature is available in the math evaluator since a few hours (in the git repo).
You can now define your own custom functions, like this :

foo : 
  -fill "
    foo(x) = 3*x^2 + 2*x + x;;
    foo(x) + foo(y);
  "

A custom function accepts up to 24 arguments, and can return a vector-valued result if necessary.
I requires still some tests to see if everything is working as expected, but if this is the case, I will probably be a nice reason to release a new version of G’MIC (1.6.9).

#13

Thank you for your hard work. Although I do not understand the things above at all, I enjoy the passion with which it is written. But I do understand that G’MIC is superb already and will be more awesome in the future.

G’MIC is like a playground with a lot of equipment to play with. The play area is that big now, that it is difficult to see when there is a new thrill ride in the park. Sometimes I feel a little bit lost. The positive thing is that I can rediscover old forgotten recreational equipment.

1 Like
(Lyle Kroll) #14

Just wanted to again tell you folks I’m stoked with this new ability. Thanks again Souphead and David for making this happen. :slight_smile:

http://gimpchat.com/download/file.php?id=20512

#15

I been reading up on this, but how I insert new value on a matrix in a loop, and the matrix resize after addition? I’m looking into a dynamic matrix or something like that.

m={w-1}

n={h-1}

k=0

l=0

do

k+=1

while $k<=$m&&$l<=$n

I’m working on replicating this code here (You seen it before on the glitch art again thread) - https://web.archive.org/web/20110813212130/http://effprog.blogspot.com/2011/01/spiral-printing-of-two-dimensional.html

My goal is to copy that code, and then use fill " i(matrix(x+y)%w,floor(matrix(x+y)/w) "

Also, I know that in C++, you could add values by v.pushback(). So, I believe this has to do with vector.

#16

I still avoid vectors and matrices and C / C++. :stuck_out_tongue:

I think recent versions allow unspecified vector sizes, so adding an element or set of elements should be simpler than C++'s pushes, pops, etc.

#17

How so? I have not really found amything about unspecified vector in gmic and push equilavent.

#18

Should be plenty of examples in the stdlib or exercises thread. Basically, declare a vector, make a for-loop and then append values to each element. Since the vector doesn’t require a predefined size anymore, you may add as many values as you desire. That is the gist of it.

What I feel more comfortable doing is making an image of values, which is essentially a matrix. The great thing about that is that this matrix is portable outside of the fill command and you could easily apply scalar, vector and element operations on it using builtin commands, which means that you could break up complex operations that you are known for into smaller chunks.

#19

From looking at the stdlib.

Ve = int() is basically a vector. But, I haven’t really found anything about appending a value in vector in the g’mic reference, and stdlib only helped with finding what’s a vector. append or ‘a’ command is the closest i can think of.

EDIT: I found poisson disk though it’s a bit confusing to figure out. But, I doubt fill will answer my problem as the original code output multiple time per “fill”, so that’s going to be a issue. fill is basically just one output in one channel at the minimum. Something like that.

Also, I found this code here - G'MIC exercises

I must look at this more because this is all very new to me.

(G'MIC staff) #20

The vector-valued variables in the math parser have a fixed size, determined at the compile-time when the variable is defined.
To manage vectors with a dynamic size, there is anyway the solution to define an image (e.g a 1x1 image), and use the functions dar_insert(), dar_size() and dar_remove() from the math_lib,just like this (dar stands for dynamic array).

foo:
  1  # Insert a 1x1 image at slot #0 (cannot be empty).
  eval ${-math_lib}" # Include functions from the math_lib
    for (k = 0, k<10, ++k,
      dar_insert(#0,k);  # Insert element 'k' at the end of the dynamic array
    );
    for (k = 0, k<5, ++k,
      dar_remove(#0,k+3);  # Remove elements.
    );

    # Display array content.
    for (k = 0, k<dar_size(#0), ++k,
      print(I[#0,k]);
    );

The image contains both the values of the dynamic array and the number of elements (stored as the last value of the image).