Store and Restore

Hello there!

(Warning: this post is mainly intended for script writers, it talks about technical stuffs about G’MIC-scripting).

I’d want to share some news about the evolution of the G’MIC scripting language here, with the apparition of two new native commands: store and restore (native means they are hardcoded in the G’MIC interpreter, so they will be available only in the next release 2.7.2 of G’MIC).

This addition comes from a thought I had after reading the question from @Reptorian on copying images in variables. These new native commands are basically a faster implementation of the img2var and var2img commands I’ve just introduced in the stdlib yesterday, as an aswer to @Reptorian’s request.

Let me quickly describe how these new commands works:

Command store[selection] takes a selection of the images in the current list, and transfer them to a binary-encoded variable (transfer meaning that after the call to store, the images are not present in the list anymore. But of course, to keep them available, you can use +store instead).

Command restore does almost the opposite : it inputs the content of such a image-based variable at the end of the image list (but it does not reset the variable content, so it’s more like getting a copy of the images stored in the variable).

The good thing is that this pair of commands relies on the same variable mechanism we had before in G’MIC, meaning the same rules apply for these image-valued variables (e.g., such variables are still local to a command, unless the variable names start with an underscore).
For instance, you can do things like:

foo : 
  sample lena,landscape 
  store[0,1] two_images   # Store 2 images in variable named 'two_images'
  if narg($two_images)    # Test if variable has been assigned
    two_images=""         # Empty variable to free stored image buffer
  fi

  sample bottles,eagle
  store[0,1] two_images   # Store 2 different images in variable 'two_images'

  (...)                   # Do what you want here
  remove                  # Empty image list

  restore two_images      # -> Restore the two images, by inserting them at the end of the image list.

  two_images={2*pi}       # Re-use variable 'two_images' to store a string

So, as you see, store and restore are just helpers to assign/access image-based content to a variable (from now, only text strings were supported for assigning variable values).

Why do we need that in G’MIC ?

Well, one use case it the following : suppose you have to write a command with an image as an argument, and do something on your selection with that argument image (e.g. blend all images of the selection with the argument image).
Then, previously, you would have written something like:

#@cli foo1 : [image]
#@cli : Alpha-blend all selected images with the specified image argument.
foo1 : check ${"is_image_arg $1"}
  pass$1 0 to_rgba.              # Insert argument image at the end of the list
  repeat $!-1 l[$>,-1]           # Loop over image selection, and keep last image for computing the blending
    +blend alpha                 # Compute blending
    reverse remove.              # Keep the 2 images in the local environment as a result
  endl done
  remove.  # Remove argument image that has been inserted with `pass`

The example above is not really complicated, but still, you see it requires you carefully manage the image list, as you have inserted an additional ‘argument’ image in the list.

Now, with store and restore, a similar command can be written as:

#@cli foo2 : [image]
#@cli : Alpha-blend all selected images with the specified image argument.
foo2 : check ${"is_image_arg $1"}
  pass$1 0 to_rgba. store. arg_img   # Store argument image in variable '$arg_img'.
  repeat $! l[$>]                    # (Classical) loop over image selection,
    restore arg_img                  # Input argument image in local environment
    blend alpha                      # Compute blending, which ends up with a single image in local env
  endl done

This doesn’t make a difference in the number of lines to write, but believe me, the second version is really easier to write !

There are still work to be done on this. Even if most of it is already functional, some things are not working as expected (in particular, the copy operator between variables arg_img2=$arg_img1 does not work right now, and it will be a bit tricky to implement. But I’ll try to do it !

2 Likes

This is excellent progress @David_Tschumperle. Thanks for asking @Reptorian.

I have been slowly developing experimental filters with multiple components. It is really hard to keep track of all of the sub-images. Being able to store them not only makes the code more accessible but it would probably reduce the amount of time lost to keeping track of which images I removed, which ones I kept and which ones I renamed. And finally, which ones I may need at the end of an extremely long multi-command filter. Just talking about it is laborious!

I hope the performance hit isn’t too much for these native commands.

I’ve worked a bit this week end and today, to improve the new commands store and restore.
Here is the current help they display:

  • Command store:

    store (+):
        variable_name1,_variable_name2,...

      Store selected images into one or several named variables.
      Selected images are transferred to the variables, and are so removed from the image list.
      (except if the prepended variant of the command '+store[selection]' is used).
      If a single variable name is specified, all images of the selection are assigned
      to the named variable. Otherwise, there must be as many variable names as images
      in the selection, and each selected image is assigned to each specified named variable.
      Use command 'restore' to get the stored images back in the list.
      
      Example: [#1]  sample eagle,earth store img1,img2 \
                 restore img2,img1
  • Command restore:

    restore (+):
        variable_name1,_variable_name2,...

      Restore images previously stored in the specified named variables, and insert them at the end
      of the image list. Note that restoring image data does not remove the data from
      the corresponding named variable (use 'variable_name=' to explicitely delete a variable content).
  • I’ve made some work to optimize the assignment of image-encoded variables, so the speed is quite acceptable right now.
  • Also, the copy operator is now functional, so you can assign an image-based variable to another variable if necessary.

In a few minutes, the pre-release version 2.7.2 will be updated with new binary packages on the G’MIC website. Let me know if you have the occasion to test this new feature!

2 Likes

Just a thought: wondering if it would be good to use store and restore. Although in English it means “to bring back” or “recover”, it could be confusing for nonnative speakers for store to be in both command names. Could be taken as “re-store”, as in “store again”. Since I used the word recover, maybe that could the name… or maybe retrieve. Thoughts?

I like the idea of having similar names for both commands, as both commands clearly work together. As a non-english speaker, I think it really helps remembering their names.

As a native spanish speaker, I have to say that I prefer reserve - recover.

Reserva = Reserve
Recuperar = Recover

But, store and restore is easier to remember as all you need to remember is store, and only add re to do something else.

Well, what I see from the Merriam-Webster dictionnary is :

  • Store : to place or leave in a location (such as a warehouse, library, or computer memory) for preservation or later use or disposal

  • Restore: to bring back to or put back into a former or original state

My understanding of that is we cannot be more close to what these commands does in G’MIC :slight_smile:
store has indeed the meaning of moving something in another place (so here, an image, from the image list to a variable), and restore brings the image back.

1 Like

No problem. Just adding to the discussion. :slight_smile:

Just for info : Today, I’ve implemented the store and restore commands, as custom commands in the stdlib.
As a result, scripts using these commands (introduced natively in 2.7.2) should work also with previous versions 2.7.0 and 2.7.1 of G’MIC (they will just be a little slower though).

So, do not hesitate to use store and restore commands in your scripts already !

  • Commands img2var and var2img have been moved to my personal update file david_tschumperle.gmic, to ensure the recent scripts from @Reptorian continue to work (but if you can replace their use with store and restore, it would be better). Not sure how long they will stay there.

  • There are anyway small differences between the native and custom versions of store and restore. I don’t think that’s going to be a problem, though.

1 Like

Have I hit a limit? This works

gmic sp tiger +store input fftpolar +.. 1 log.. store mag,phase restore input,mag,phase

but when I input a larger image such as heal_mushrooms.jpg, I hit a wall

[gmic] *** Error in ./store/*if/*substitute/uchar2base64/ *** Command 'eval': gmic<float>: Function 'copy()': Out-of-bounds image pointer (length: 2, increment: 1, offset start: 92434578, offset end: 92434579, offset max: 92434575).

Yes, that’s one difference with the ‘native’ versions of the commands.
Your example works with 2.7.2_pre currently.

You mean the native versions can handle this? I am confused which ones are native and which ones aren’t…

@David_Tschumperle I figured it out, and changes work on my tiled form filter. If you’d like, you can remove the arg2var and var2img.

Yes, that’s it.
Native versions of store and restore are basically there is you are using G’MIC 2.7.2.
Before that version, the limited custom versions of these commands applies.

Oh, I think I know what you are saying: native overrides stdlib. Since I haven’t updated core yet, I am using stdlib's.

You could check https://gmic.eu/files/prerelease/, where a Windows build of G’MIC 2.7.2 is available for testing.

After thinking about it, it would be interesting to use store and restore command to replicate this -

Is it feasible? Right now, all I can think of is a gui filter that uses text input. Rather tiring to do it manually.


@David_Tschumperle

I just have one issue with store and restore. I really don’t want to see this in preview when using code[local]. The command is basically for reverse engineering gradient map.

Same thing with apply_tiles.