I have spent all day scouring the tutorials pages, stdlib.gmic, and the forum looking for documentation or examples of how to write scripts. I have managed to find a few things in the stdlib, but I have no idea how much of anything goes together. I am specifically looking for how to manage passed in arguments and what kind of image list the command has access to (things seem to be nonsensically out of bounds). Is there any place that gives a cohesive tutorial on how to write scripts?
@grosgood has done a great job at writing tutorials for creating scripts.
It starts here : Introduction
2 Likes
$-expressions, predefined in custom commands, govern much of the translation from arguments users may write after your custom command and its body. Adding Custom Commands furnishes a terse list of them. You have already seen some of these: $0, $1, $2,… The very first resolves to the name of the script, $1 resolves to the first argument, $2 the second, and so forth. In addition to these, there are other pre-defined $-expressions that furnish information about the command line: $# furnishes the maximum number of known arguments. Some G’MIC commands are more-or-less dedicated to argument processing -check furnishes the means to write argument tests; -skip permits some arguments to be optional (but they need default values, set with this idiom ${2=3}. That is, the second argument, if not expressly given by the user, defaults to the value 3. 3D Bouncing Balls demonstrates the writing of custom commands through iterative versions.
Commands operate on every image on the list, unless constrained otherwise. I call such constrained commands ‘decorated.’ These decorations are the square brackets placed to the right of the command (no space!) that have a terse notation written within. That notation constitutes a mini-language in its own right. Command Decorations examines this notation. Their aim is to ‘select’ images for being operated on by the command, so these decorations are commonly called ‘selections.’ Tiny little pips .. constitute a shorthand form of command decorations; these select the last (.), penultimate (..) and third from the last (...) image, useful because the final images on the list are the most frequently selected.
Hope this helps.
4 Likes
Addendum:
Here’s a common difficulty: you do not have a G’MIC vocabulary. You cannot connect your visual goals with some pipeline of G’MIC commands.
That’s just the way it is. We all start at that point. In tutorial land, the question of interest is: “How best to get people quickly past that point?”.
Theory: Finger exercises. Conceive simple visual goals; challenge yourself to come up with pipelines that realizes those goals.
[Aside: My choice of the phrase ‘finger exercise’ is deliberate. You cannot play Beethoven’s Sonata 32, op. 111, until you’re past Chopsticks. You must have muscle memory. In your fingers, in your feet, verily — in your whole body. Muscle memory has to be instinctive so you can focus on the really hard part: interpretation.]
Start with simple — or what may seem to be simple — visual goals. For example, this morning, riding into Manhattan on the subway, I happened to glance down at the linoleum floor of the car and asked myself “How, in G’MIC, do I make a texture like this floor tile?”
It took me a half hour, spread over the morning, while waiting for various meetings to start, to realize this visual goal — somewhat. The dreary color scheme is spot on. But the little color bits embedded in the base linoleum had different shapes: triangular, square, irregular polygons — like bits of chewed-up rubber balls. Look closely and my results: the “rubber bits” all have the same shape, the same size. A miss. That’s the way it is with these finger exercises. You get part of it right; part of it is a miss. You then find yourself pulling in other G’MIC commands to get it all right. That’s how you build vocabulary.
At the moment, you don’t have much of a vocabulary. You can see your visual goal, but you are not sure how to build the pipeline to get there. Here are some vocabulary building tips.
-
Scroll through the Technical Reference. Don’t read it. Just look at the pictures. You might see something that’s half-way like your visual goal. If you do, stop and read about the command(s) that produced the picture.
-
Write test jigs so that you can quickly assess the effects of command arguments. Here’s an example:
testcmd:
count=10
-repeat $count
-local[-1]
# Command being tested:
+blur. {($>+1)},0,1
-done
-reverse[-2,-1]
-done
-move[-1] 0
Take this to be an exercise in writing a custom command. This custom command exercises commands. The command being exercised here is in a repeat loop. We are testing blur by repeatedly changing one argument. The bits in the curly braces constitutes a small mathematical expression. There is a predefined substitution variable, $>, which G’MIC substitutes with the current loop count. So, loop on loop, the argument gets larger. This is the results of testing blur with different initial arguments:
Play this game with any number of commands, and any number of arguments for each command.
- Get Git. Really? A version control system? Why? To snapshot your work as you go along. If you make a mess, you can backup to the last snapshot that worked. Every commit makes a snapshot. Commit often. If you want to try different things, you can make branches at any one of your commit points. Keep branches forever if you want: variations on a theme. Git repositories aren’t just for large projects. They work for tiny projects too. Here’s how to get started creating a repository for developing
testcmd:
gosgood@lydia ~ $ git init testcmd
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
Initialized empty Git repository in /home/gosgood/testcmd/.git/
gosgood@lydia ~ $ cd testcmd
gosgood@lydia ~/foo $ touch testcmd.gmic
gosgood@lydia ~/foo $ git add testcmd.gmic
gosgood@lydia ~/foo $ git commit -m "Starting to write the testcmd command"
[master (root-commit) 053f925] Starting to write the test command
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 testcmd.gmic
Now you can edit the empty testcmd.gmic file. Perhaps copy the jig given above into testcmd.gmic. Change things. Maybe even break things, deliberately, just to see how the pieces fly. Then perform git add --all followed by `git commit -m “.”. You may wax prolific and write a little log note to yourself instead of the sole “.” but you don’t have to. The important thing is, commit often. Every commit is a snapshot. Screw up and you can back up to the last working snapshot. Later on, you will want to try different approaches. Create branches. You can create a branch at any commit point, even one made days ago. If you develop something good in a branch, you can merge it back to the master (initial) branch. Working in a repository can save you from an enormous amount of grief. Before Git, we’ve all worked into the wee hours, realized that we’ve screwed up, and — even worse — can’t remember how the the script was before we screwed up. No commit points. No way to travel back in time, to 9 PM, when we were still more-or-less awake. So yes. Get Git. It’s in your distro, and its available for Windows.
Your finger exercises work best when your visual goals are immediate; the number of gmic commands are likely to be small. Scale up as your vocabulary and confidence grows. Look around you every day, especially textures. Ask yourself how you can get that texture through G’MIC.
How many finger execises? Lots and lots!. Maybe three or four hundred per year — one finger exercise per day. Ah! You mean you have a day job??? How unfortunate. But I suppose you have to eat, pay rent and such. Then try for one finger exercise a week. Fifty or sixty in the next year. Any slower than that and — don’t take this unkindly! — you’re not serious. That’s also fine. One of the best bits of education I had ever received, back in college days, and — late again! with one assignment or another — was the professor’s sigh. And then he said: “You know? People always find time to do what they really want to do.”
Procrastination is instructive. It’s how your head tells you who you are.
4 Likes
Thanks for the response!
My main question here is when using a selection as an input on a decorated command, the image selection seems to be out of bounds. Here, calling the following command produces an error:
gmic differ.gmic repeat 2 +lorem 200,200 done color_difference[0] [1]
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input custom command file 'differ.gmic' (1 new, total: 4568).
[gmic]-0./*repeat/ Input random image of size 200x200.
[gmic]-1./*repeat/ Input random image of size 200x200.
[gmic] *** Error in ./+color_difference/deltaE/ *** Command 'pass': Invalid selection '[1]' (contains index 1, not in range -1...0).
differ.gmic:
+color_difference:
check ${"is_image_arg $1"}
rgb2lab
+deltaE $1
lab2rgb
The error message leads me to believe that the custom command only has access to the decorated selection; therefore, any passed in indices would be out of range? I know this is not the case, because I can use other commands that use guide images (bilateral). My plight is that documentation about what is actually happening inside of commands is very spread-out across commands and all of the examples/tutorials deal with isolated parameter-less scripts. Is there a place that I can start that gives me a good base knowledge of how to make a script rather than how to make an effect?
I apologize if this comes of as a little whiny.
Thank you for this treasure trove.
Finger exercises are definitely something I want to do and the Git advice is something I should be implementing everywhere.
Actually, I owe you thanks for sharing your trials and tribulations. We who have been writing G’MIC scripts for twelve or thirteen years have forgotten how we took our first steps. As one of the principle tutorial writers, I’m very keen on what G’MIC looks like to newcomers. What kind of mental models do newcomers build inside their heads about how G’MIC works?
That’s right. There are no extensive tutorials on how to get inside a script and observe its inner workings. My first foray into this topic is a cheat sheet: Cheat #4: You’ve Moved Fast and Broken Things. Now What?.
Give it a read. From that, here’s a few tips on G’MIC debugging:
TIP 1: Use -verbose to make your custom command more (or less) noisy. See the verbosity section of the cheat sheet for background and some technique. For now, I am going to make your invocation of color_difference a little more self-revealing. Here is how I’ve modified your command line:
gmic differ.gmic repeat 2 +lorem 200,200 done verbose + color_difference[0] [1]
That is, before I invoke your custom command, I up the verbosity level so that your custom command can emit log lines that are otherwise supressed.
TIP 2: Hijack the display command for single step debugging.
To that end, I’ve modified your custom command:
differ.gmic:
+color_difference:
check ${"is_image_arg $1"} d ,
rgb2lab d ,
+deltaE $1 d ,
lab2rgb d ,
Every other command is the display command (shortcut: d). The optional comma is to indicate that I’m not passing any arguments to display. The effect of this to execute one command of your script, then stop to display the image list. For all practical purposes, you are single-stepping through your custom command. At each step, a display command stops further execution and provides you with the opportunity of examining the display list and inspecting what your command is doing.
With your script and command line modified, lets give it a run:
$ gmic differ.gmic repeat 2 +lorem 200,200 done -verbose + color_difference[0] [1] -verbose -
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input custom command file 'differ.gmic' (2 new, total: 4568).
[gmic]-0./*repeat/ Input random image of size 200x200.
[gmic]-1./*repeat/ Input random image of size 200x200.
[gmic]-2./ Increment verbosity level (set to 2).
[gmic]-1./+color_difference/ Display image [0] = 'lorem', from point (100,100,0).
[0] = 'lorem':
size = (200,200,1,3) [468 Kio of float32].
data = (192,189,186,182,179,175,173,172,178,183,190,193,(...),5,5,3,0,1,1,0,0,0,0,0,0).
min = 0, max = 255, mean = 138.866, std = 74.2498, coords_min = (89,45,0,2), coords_max = (190,26,0,0).
[gmic]-1./+color_difference/ Convert color representation of image [0] from RGB to Lab, using the E illuminant.
[gmic]-1./+color_difference/ Display image [0] = 'lorem', from point (100,100,0).
[0] = 'lorem':
size = (200,200,1,3) [468 Kio of float32].
data = (88.9174,88.8173,88.2612,87.9715,87.255,86.9595,86.8896,86.6923,87.215,88.0006,88.3893,88.3279,(...),99.7119,99.7119,106.114,121.207,115.611,115.611,120.914,120.542,120.542,120.542,120.542,120.542).
min = -8.29922, max = 123.04, mean = 41.4644, std = 34.2813, coords_min = (65,0,0,2), coords_max = (199,172,0,2).
[gmic]-1./+color_difference/ Compute the CIE DeltaE_2000 color difference between image [0] and image[1], with to_Lab command 'srgb2lab'.
[gmic] *** Error in ./+color_difference/deltaE/ *** Command 'pass': Invalid selection '[1]' (contains index 1, not in range -1...0).
When we enter the scope of color_difference single-stepping begins: and the first hint of trouble rears up. First, in this new scope, there is only one image being displayed by our debugging display command, not two. That’s not good. Can’t difference between two images when there is only one on the image list. There’s another indication that there is only one image on the image list: See how [gmic]-2. notation transitions to [gmic]-1 on the transition into color_difference scope? G’MIC’s shell log indicates what images are visible on the image list. On the scope transition, the visibility count dropped from two to one.
What about $1? Doesn’t that reference the second image? No — there is a misunderstanding here. The $-expression $1 only resolves to a string that looks like a selection decorator, that is $1 ⇒ [1], literally: open square bracket, numeric character ‘1’, close square bracket. Why? Well, that bracketed numeral is what was actually, literally, written in the argument list to color_difference. G’MIC is not imaginative enough to interpret the argument list in any other way than the literal string characters that are written there. That’s all what “is_image_arg $1” tests: It resolve to True when an argument looks like a selection decorator.
So how does one actually pass an image argument into a custom command scope? For that, use the pass command. So, let’s modify your custom command this way:
+color_difference:
check ${"is_image_arg $1"} d ,
pass$1 0 d , # New command!
rgb2lab d ,
+deltaE $1 d ,
lab2rgb d ,
When G’MIC is resolving all the command line substitutions, pass$1 0 ⇒ pass[1] 0. After scope entry, with the pipeline interpreter working its way down the color_difference pipeline, it encounters the pass command. It is the nature of the pass command to interpret its selection decorator in the parent scope, So pass[1] 0 may be read as "Take the image with the index 1 in the parent scope and duplicate it into color_difference scope. Argument 0 of the pass command makes a full copy, requiring more image storage, but making a new, independent image.
Now you have both images present in your command’s scope. Run it and you’ll get your differences as expected. Then you can delete all the d , “break points” and remove verbose from your command line and get on with your next adventure.
Have fun.
2 Likes
Thank you so much for your patience. This some great instruction
For me, I’m coming from a background in amateur programming; very clean, clear,terse syntax. I’m used to:
def do_this_with(thing_one, thing_two="something")
I now understand how to do this with gmic, but the connections to conventional programming should be more readily made for introductory scripting (under the warrant that anyone crazy enough to jump down the gmic rabbit hole would have at least some programming knowledge).
I’m also used to being able to read a line of code and have it describe to me what it’s doing; with gmic, I have to decipher. I get that eventually it will become familiar and I won’t have to constantly reference the shortcut list and technical reference (departure from tutorial land), but for comparison, right now, having had no experience with image processing pipelines, gmic is like going from drag’n’drop to c++, minus the countless bootcamp video series. Don’t get me wrong, your tutorials are immensely helpful and enjoyable, but trying to self teach off of written instructions is always difficult.
Thank you again for your patience and heaps of advice, I will utilize them in my endeavors moving forward.
1 Like
I want to point that G’MIC can be said to have 2 languages in one. Eval, and fill, and math expression have their own quirks and it use it own language. Math parser language has a lot of things in common with C++ syntax like “;” for end of line, the use of macros (G’MIC solution to function (and this comes with limitations in which requires pushing stack to solve)), ++/-- things and yes order matter. But, non-math parser thing comes first.
You can also use exec command to execute Python script and so, but chances are you’ll never need this.
2 Likes
And…
In my other life, I work for New York City (Housing Preservation and Development, in particular), where code writing rules are enforced: these demand “very clean, clear,terse syntax” so that the code approaches this ideal of “describing what it is doing.” That particular environment demands such a use-case. Software developers come and go; the code they leave behind has to be readable by an influx of new-hires. ‘Sensible programming’ puts a premium on these readability ideas.
But G’MIC comes from a research environment — and very different use-cases prevail. Sometime last December, in a riff off of @David_Tschumperle 's How to write clean G’MIC code ? I veered into a post on G’MIC’s raison d’être: as stenography for graphical pipeline builders. A bit tongue-in-cheek of me, perhaps, but I can make my case: David (and colleagues) are very much involved in testing graphical ideas. From David:
So a tool that gets past the junk-thinking very quickly is desirable. Above all, that puts a premium on terseness, and, in a priority stacking, it’s terseness above clarity, by very deliberate design. The aim is to test ideas without the damn code getting in the way, without exasperating code ⇒ compile ⇒ test cycles, without leaving the shell. So syncopation rules: status ⇒ u, append ⇒ a, fill ⇒ f, and — my favorite! — input ⇒
As with most design decisions, there are prices to be paid for a particular priority stacking:
Amen. The newcomers have to climb a steep learning curve.
However, there are silver linings. So while I am inventorying the various shortcomings in Tutorial Land, here is one of the most significant: “The Power of the Colon ( : )”, as in:
< a nice, apt, and memorable name > : < hundreds of lines of terse, syncopated G’MIC code, full of weird, incomprehensible constructs that — dammit! — does very useful things, now if I can only figure out how it does it. >
The colon “operator” lies at the heart of custom command definitions. It is G’MIC’s language extension mechanism. Among other things, one may write a comprehensible left-hand name to represent a right-hand horror show. You have already recounted your ventures into gmic_stdlib.gmic and have probably noticed that in its twisty, darkened hallways there are these one-line custom command definitions that give rise to the ‘short-cut’ names for common commands:
You can push that pea in directions other than syncopation, come up with custom command names that better suit your taste and put them in your local G’MIC extensions file: %AppData%\user.gmic (Windows) or ${HOME}/.gmic (Unix-like). In that file, you could even be so bold as to override the default, but empty, cli_start custom command. G’MIC references this particular command, automagically, on every startup. It’s a hook. You can redefine cli_start to include various *.gmic files and introduce site-specific constructs without using -command inclusions. These syncopation tools, used in the service for People Who Hate To Type, are amendable to other kinds of language extensions.
The only cautionary that I should make is that in sharing environments such as this Discourse server, the canonical G’MIC must rule; it can be hard enough for people to figure out what other people are doing without the babel of regional dialects cropping up.
That’s it for now. Good luck with finger exercises and spelunking gmic_stdlib.gmic.
3 Likes
When I first started g’mic, I spent two solid weeks on it (yes, daily) before I could do even basic stuff. Most of it was just experiment and looking at scripts written by others. In style, it has something in common with maths; lots of symbols and layout with concise meaning that you just have to memorise.
Three things were a mystery: the image stack, the “substitution” mechanism (pretty much macros) and how images themselves worked.
Some old pages to help with the basics have also landed on the github wiki, they need to be updated but are still useful. Scroll to the “Technical Documentation” heading and you may find relevant parts…
3 Likes
I spent way more than that before I get to my first filter. I feel like I’m still learning.
1 Like
gmic 128,128,1,1,255*x/(w-1)
which produced:
A ramp. That was the first exercise which gave me a sweet lift of victory. It was, I think, mid 2012. Hitherto, it was random stabs, extending back months. Since I didn’t fully grasp at the time that I had to prepare output, a lot of the results were not usable in Gimp, which frustrated me to no end (but I can see them in the display window! Why not Gimp?). This exercise was procedural — an image generated from nothing at all! That intrigued me no end. From that victory, I began experimenting more consistently — and I started a notebook, which I put on my own web site. This was partially to keep track of how I did what I did, and to help others who might be curious about the tool. The notebook about the ramps eventually evolved into this tutorial, initially hosted at my own website. Somewhere during 2013 @David_Tschumperle stumbled across my notebooks and the rest is tutorial history. By then — first contact — I was on a roll. The few little victories that I had lay the foundation for the viral stage of learning, where results snowball into other results. I am still on that roll.
I think that is why I want to pursue this notion of “finger exercise.” They lead to the first, little victories that, for me, at any rate, got me off the dime. It is missing in Tutorial Land; they are the first discoveries that open doors. I’d like to capture those first “Ah, ha!” moments, if I can.
5 Likes