Hello.
I want to keep you up to date on the latest developments I’ve done on the G’MIC standard library.
In the last few days, I’ve indeed added a few new commands (and removed quite a few too!), for two distinct topics:
1. Image resizing: rescale2d and rescale3d (shortcuts rs and rs3d).
Looking at the G’MIC reference documentation, I realized that there were a lot (too many!) different commands for resizing images. The most generic command for image resizing is resize, but it’s not practical to use if you want to preserve the aspect-ratio of your images (which is something you often want!). To resize an image while preserving its ratio, then you had the choice between :
-
For 2D images :
resize2dx,resize2dy(a.k.a.r2dxandr2dy) : Ratio-preserving resize of images by specifying either the width or height of the resulting image.resize_ratio2d: Ratio-preserving resize of images by specifying either a minimal or maximal size.resize2din,resize2dout: Basically same asresize_ratio2d(I don’t even remember why I defined those).
-
For 3D images :
resize3dx,resize3dyandresize3dz(a.k.a.r3dx,r3dyandr3dz) : Ratio-preserving resize of images by specifying either the width, height or depth of the resulting image.resize3din,resize3dout: Basically same as their 2d counterpart, but for 3d images.
So there were at least 10 different commands for this ratio-preserving image resizing job!
So I thought it was time to do something about it 
To simplify things, I’ve then added two commands in the stdlib, namely rescale2d and rescale3d (shortcuts rs and rs3d) which are capable of doing absolutely everything that could be done with the old commands (and more), and both having equivalent arguments, so it’s easier to remember.
For instance, this is the help of rescale2d:
$ gmic h rs
rs:
Shortcut for command 'rescale2d'.
rescale2d:
_width[%]={ 0:any | >0 },_height[%]={ 0:any | >0 },-1=<_interpolation<=6,_mode={ 0:inside | 1:padded-inside | 2:outside | 3:cropped-outside }
Resize selected 2D images while preserving aspect ratio.
'interpolation' can be { -1:status only | 0:none | 1:nearest | 2:average | 3:linear | 4=grid | 5=bicubic | 6=lanczos }.
When 'interpolation==-1', image size is actually not modified, but the size that would have
been used for the last selected image is returned in the status value.
Each resized image size is computed according to the specified 'mode':
* If 'mode==0', image size is at most '(width,height)'.
* If 'mode==1' or 'mode==3', image size is exactly '(width,height)'.
* If 'mode==2', image size is at least '(width,height)'.
(equivalent to shortcut command 'rs').
Default values: 'width=height=0', 'interpolation=2' and 'mode=0'.
And basically, it can be used for these everyday tasks:
- Resize a 2D image according to a target width of 128:
$ gmic image.jpg rs 128(eq. toresize2dx). - Resize a 2D image according to a target height of 128:
$ gmic image.jpg rs ,128(eq. toresize2dy). - Resize a 2D image so that it fits into a 128x256 area:
$ gmic image.jpg rs 128,256(resulting image size can be then smaller than 128x256, eq. toresize_ratio2d). - Resize a 2D image so that it fits into a 128x256 area, and add zero-padding to match the specified size :
$ gmic image.jpg rs 128,256,2,1(so here, resulting image size is exactly 128x256, also eq. toresize_ratio2d). - Retrieve only the image size to make it fit in a
128x256canvas :$ gmic image.jpg siz=${"rs. 128,256,-1"}.
And it works the same for 3D images, with rs3d (with an additional depth argument).
Two commands were necessary for 2D and 3D images, because a 3D image upscale with ratio preservation may increase the number of slices of an image, which is something we never want for 2D images.
rs is very useful, and I’ve put it everywhere in the stdlib when that was possible. And all “older” commands for aspect-ratio preserving resize have been moved out of the stdlib (in my community .gmic file, so that they are still available for older filters and commands).
But I recommend you use it, it’s really handy!
2. Commands for smooth image compression / decompression: compress_to_keypoints and decompress_from_keypoints.
With the new commands compress_to_keypoints and compress_from_keypoints, I wanted to revisit the work I did for the (lossy) compression of color LUTS some years ago (see my publication in the SIAM SIIMS journal for details), but that would work for more generic images (2D or 3D, with an arbitraty number of channels).
The idea behind the compression algorithm is to find a set of keypoints so that decompressing is just interpolating the values of those keypoints in the 2D or 3D space, either using RBFs (hello @grosgood!) or Diffusion PDE’s.
In G’MIC, a set of keypoints is represented by a single column vector-valued image that contains the keypoints coordinates and their values, as well as a small header (the first two pixels) that tells about the original image size and value range.
The help for compress_to_keypoints is:
$ gmic h compress_to_keypoints
compress_to_keypoints:
_method,_max_keypoints>=0,_err_avg[%]>=0,_err_max[%]>=0,_"err_command"
Compress each of the selected images into a set of keypoints that can be further decompressed using command decompress_from_keypoints.
Beware: This type of compression is effective only for images with very smooth content.
'method' can be { 0:PDE | 1:RBF }. Add '2' to 'method' to enable removal step.
* 'max_keypoints' is the maximal number of keypoints generated by the compression method. If 'max_keypoints<0', the removal step is not done when
number of maximal keypoints has been reached. 'max_keypoints=0' means 'no limits'.
* 'err_avg' is the desired average compression error.
* 'err_max' is the desired pointwise max compression error.
* 'err_command' is the code of a command that inputs the two images '[reference]' and '[compressed]' and compute a single error map as a last image.
Defaults values: 'method=3', 'max_keypoints=0', 'err_avg=1%', 'err_max=5%' and 'err_command=-. [0] norm.'
This command is quite generic, e.g. you can specify your own error-estimation function (for CLUT compression, I use for instance the DeltaE_2000 perceptual distance, as I deal with RGB colors in that case). The compression process is demanding, it can take several minutes/hours to complete.
The smoother the image, the fewer key points you’ll need to represent it.
Once your set of keypoints is computed, just use decompress_from_keypoints to retrieve an estimation of the original image. The compression algorithm is lossy, so you won’t get exactly the same image, but something close enough if you chose the arguments of compress_to_keypoints wisely. The help for the decompress_from_keypoints is:
$ gmic h decompress_from_keypoints
decompress_from_keypoints:
_width>0,_height>0,_depth>0 |
(no arg)
Decompress selected sets of keypoints as images (opt. of specified size).
A set of keypoints is defined as a vector-valued image, such that:
* The first pixel is a vector which encodes the '[ Width,Height,Depth ]' of the decompressed image.
* The second pixel is a vector which encodes '[ Min,Max,Use_RBF ]', where 'Min' and 'Max' defines the value range of the decompressed image, and
'Use_RBF' tells is the decompression scheme must use RBFs ('Use_RBF=1') or Multiscale Diffusion PDE's ('Use_RBF=0').
* The remaining pixels define the keypoint coordinates and values, as:
- '[ x_k,y_k,z_k, v1_k,...,vN_k ]' for a 3D target image of N-valued vectors.
- '[ x_k,y_k, v1_k,...,vN_k ]' for a 2D target image of N-valued vectors.
- '[ x_k, v1_k,...,vN_k ]' for a 1D target image of N-valued vectors.
where the coordinates 'x_k', 'y_k' and 'z_k' are defined respectively in ranges '[0,Width-1]', '[0,Height-1]' and '[0,Depth-1]'.
If the 'width', 'height' and 'depth' arguments are provided, they define the size of the decompressed image, : overriding then the original image size
'[ Width,Height,Depth ]' defined in the keypoints header.
Here is an example of compression/decompression of a smooth 2D image:
$ gmic sp colorful,256 b 3% +compress_to_keypoints. , +decompress_from_keypoints.
Here, colorful is the original color image, colorful_c1 is the set of keypoints computed by compress_to_keypoints (here, 407 keypoints), and colorful_c2 is the result of decompressing this set of keypoints with command decompress_from_keypoints.
These commands are really interesting when the image data you need to compress contain very smooth transitions. That’s clearly the case for 3D CLUTs in general, and I’m currently trying to recompress all the CLUTs available in G’MIC with this new compress_to_keypoints commands, as it seems it works slightly better than what I used before (so I could try being less lossy!).
But don’t try with highly detailed images, it won’t work (or maybe with a number of keypoints that is equal to the number of pixels in the input image).
Fun derivations:
As sets of keypoints are stored as 1-column images, we can somehow play with these compressed representation of images.
For instance:
- Generate a 512x512 image of random color gradients :
$ gmic "(512^512^1)" "(0^255^1)" 1,30,1,5,"[v([511,511]),v([255,255,255])]" a y decompress_from_keypoints
- Generate a morphing between such color gradient images:
foo :
repeat 10 { l[] { "(256;0^256;255^1;1)" 1,50,1,5,"v(255)" a y decompress_from_keypoints. 128,128,1 } } [0]
morph 32,0.1 rm.
o video.gif,20
- Generate an animation between two compressed keypoint-based representations of two different portraits, by linearly interpolating the keypoints sets:
Voilà ! These are just some of the new features introduced in G’MIC recently. More fun to come!
!).