# Image transformation using matrix

Hi,

I am probably missing the right operation to use but I was not able to find how I can apply a 3x3 matrix to transform an image. This is just an affine transformation matrix. I can decompose it into a translation and rotation and use the shift and rotation commands but I would rather apply the full matrix directly.

Please let me know what would be the right command to use. I searched!

Thanks,

• Alex

Usually a 3x3 matrix is for changing the primaries, etc. For rotation and the like, It is about resorting pixels. For that you would need something like this example (note it is for MATLAB but it is just to make the point): Image rotation by Matlab without using imrotate - Stack Overflow.

GāMIC reference documents:

There may be a better way, but you can create a vector field using matrix and coords, then use `warp`:

`gmic sp cat +f "[0,1,0,1,0,0,0,0,1]*[x,y,z]" warp.. .,0,1,0 rm.`

@grosgood wrote an old tutorial on it: `https://gmic.eu/oldtutorial/_warp`.

Iām guessing this would be the relevant article: Image Geometric Transformation In Numpy and OpenCV | by Daryl Tan | Towards Data Science

I think itās possible to convert it, but Iām not in the mood to look at it for now. Iāll see to it later when I can.

@garagecoder touched the bit on image rotations; another interesting area for affine transformation matrices is colorspace rotations ā an easy idea to take in if one is comfortable of thinking of color as forming a three dimensional space with red (neĆ© x), green (neĆ© y) and blue (neĆ© z) axes.

Some recent tutorials which play with this concept of color space rotations:

GāMICās math parser serves as a nice laboratory for concocting 3D affine matrices on-the-fly. `eye(3)` produces a 3D identity matrix and `rot([ax, ay, az],aĀ°)` produces a matrix that rotates points for an angle of a degrees around the axis ax, ay, az (āĀ°ā is a GāMIC degrees ā radians converter; it operates on its left hand parameter, in degrees, and replaces the expression with the radians equivalent). `mul(A,B,ccount)` multiplies matrices; `ccount` furnishes matrix widths. As always, matrix multiplication is non-communicative. translating and rotating is not the same as rotating and translating. But you knew that. See Vector Calculus.

If you wish to write affine transformation matrices for position vectors written in homogeneous form `[x,y,1]` then here is an affine matrix which first translates points 3 units in the x direction, -7 units in the y direction, then rotates the results 43 degrees counterclockwise around the z axis:

`````` \$ gmic foo={am=eye(3);am[2]=3;am[5]=-7;mul(rot([0,0,1],43Ā°),am,3)*[0,0,1]} echo \$foo
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Set local variable 'foo=6.9680496252950013,-3.073480831146699,1'.
[gmic]-0./ 6.9680496252950013,-3.073480831146699,1
[gmic]-0./ End G'MIC interpreter.
``````

so the origin of the surface at homogeneous point (0,0,1) translates and rotates to ā (6.968, -3.073, 1)

1 Like

Thanks for the simple example. I am aware of the syntax but have no idea how to put it together.

Combining @garagecoder and my previous into one may be closer to your original ask: you want to twist and toss a cat. Donāt do this with a real cat; skin integrity may suffer. The game is to fill a warping image with new coordinates. A pixel in the warping image at location x,y contains the coordinates of where the pixel in the cat image at location x,y is to go in the final output image. Here, we fill the warping image with coordinates calculated from an affine transformation matrix, the transformation matrix taking the x,y pixel location as the initial position vector and producing the transformed position ā typical affine transformation matrix work. To make this example a little more interesting, the final affine transformation matrix, `fin` is a composition of (1) a translation matrix, to bring the center of the cat image to the origin, (2) a rotation matrix, twirling the cat around the origin, and (3) a second translation matrix to throw the cat more-or-less back around the center of the image, but not quite in the same place. `expand_xy` is handy for furnishing some working room.

`xfrmfun.gmic`:

``````xfrmfun:
+f. "begin(
xlt0=eye(3);
xlt0[2]=w/2;
xlt0[5]=h/2;
fin=mul(rot([0,0,1],73.25Ā°),xlt0,3);
xlt1=eye(3);
xlt1[2]=-1.5*w/2;
xlt1[5]=-0.75*h/2;
fin=mul(fin,xlt1,3)
);
p=fin*[x,y,1];
[p[0,2,1],0]"
warp.. .,0,2,2
rm.
``````

Thereās a bit of trickery toward the end of the math expression. In most of the transformation pipeline, we are manipulating homogeneous position vectors `[x,y,1]`. When it comes to dropping a pixel into the warp image, we extract the `x,y` part, leaving behind the one, then compose a new vector, a pixel with channel 2 set to zero always; that, then, drops into the warp image.

Usage:

``````gmic xfrmfun.gmic sp cat,256 expand_xy. 200,0 xfrmfun. o. 'throwncat.png'
``````

Have fun!

Thanks everyone for the suggestions and codes. It seems āwarpā is what I need.

I do have a 3x3 matrix already, in homogeneous coordinates. I entered in @garagecoder code but it didnāt quite work as expected. Maybe I am not getting the order of values correct. Shouldnāt the matrix, āfinā in @grosgood code, be the sequence of rows from my original 3x3 affine matrix?

• Alex

Well, I was far too brief in the explanation - with a displacement field, the vectors being manipulated reference a source location to āpullā a pixel from. I think youāre looking for manipulations of the source pixel coordinates themselves, which is more like a āpushā method.

Anyway, I donāt want to mislead so Iāll be reading the other suggestions myself before I come up with anything furtherā¦

Thank you @garagecoder. What I have in mind is the transformation [xā yā 1] = T*[x y 1] such that the new position (xā, yā) contains the pixel values/intensities of the source location (x, y). I think this is the āpullā version you mentioned and coded, and the one in @grosgood code.

In that case, Iām quite sure that would be pushing pixels to a new location. The `warp` command will pull a pixel value from a location given by (x,y,z). But thatās not a huge problem - you only need invert the matrix for it to work with warp:

``````gcd_affine :
if \${"is_image_arg \$1"} pass\$1 1 else i \${^0} fi
invert. +f[0] "[x,y,1]" mix_channels. .. rm..
sh. 0,1 warp[0] .,0,1,0 k[0]
``````

`gmic sp gmicky gcd_affine "(0.707,-0.707,256;0.707,0.707,-128;0,0,1)"`

Notice thereās no handling of the origin or image extents there, a properly written command would need additional parameters and ability to work with a list of images.

2 Likes

Thanks @garagecoder! This is exactly what I had in mind. Having a black background for uncovered pixels is fine and I donāt think we need origin in my case. This is the solution I can use.

Would warp work when we have sparse white points on a black background, like salt-and-pepper noise? I have a suspicion that interpolation would make the transformed image all black, need to check.

The black pixels will move with the white. The transformation would be for the whole image array.

I think for most purposes itās fine - you can choose between bilinear and bicubic interp with `warp` as well. Thereās always some loss of information at certain angles on a raster grid, the choice is what you want to lose.

Obviously using the original input each time rather than repeated applications is bestā¦ if sharp individual pixels is what you need, youād be better with a āpushā method such as a point cloud.

Thereās the concept of rotsprite if one needs to rotate pixel art image, but thatās really for another subject even if tangibly related. For photographs and stuff, @garagecoder is on point.

@garagecoder To clarify for our intrepid readers, please explain what you mean by push and pull. It isnāt martial arts is it?

Maybe a simple example would be better than 1000 words.
From what I understand, if I is your input image, J, the warped image, and U = (u,v) your warping field.

• Push : J(u(x,y),v(x,y)) = I(x,y)
Effect of push mode with `warp`:
``````\$ gmic sp cat,128 100%,100%,1,2,4*[x,y] +warp[-2] [-1],2,0
``````

Here, each pixel (x,y) of the original image I is pushed at the locations (4*x,4*y) in the target image J (so it creates a grid of non-zero pixels).

• Pull : J(x,y) = I(u(x,y),v(x,y))

Effect of pull mode with `warp`:

``````\$ gmic sp cat,128 100%,100%,1,2,4*[x,y] +warp.. .,0,0,0
``````

Here, each pixel (x,y) of the target image J is pulled from the locations (4*x,4*y) of the original image I (so it creates a 4x smaller version of the original image).

Note that the warping field is the same in both examples.

1 Like

Thanks for the examples. I see that push (2) and pull (0) are the mode parameters for warp and probably there is no need to invert the affine matrix as @garagecoder has done?

I have both point clouds and full images to warp so pull and push will be needed. These would be very handy as native commands - not to add extra coding work but to enrich the list of commands.

It depends. On the first example above, if you need a ācontinuousā image as a target, itās probably better to use the āpullā mode instead, by inverting the warping field:

``````\$ gmic sp cat,128 100%,100%,1,2,0.25*[x,y] +warp.. .,0,0,0
``````

I would say that if itās possible to invert the warping field, itās always better to use the āpullā mode (youāll never create āholesā in your target image).
Itās not always possible to invert it though.