G'MIC exercises

exercise
#1

Learning how to write commands isn’t easy. I think I need to do some exercises. This thread will be a collection of them.

Minimum noise B&W conversion
(http://www.guillermoluijk.com/elmomentodecisivo/bwalgorithm.R)

I am not getting it right; I probably read the code wrong. This is what I have, ending up with more noise rather than less!

gmic s.tif n 0,255 to_gray

dcraw -v -r 1 1 1 1 -o 0 -4 -T -O s.tif SnowOnSpiderWeb.cr2
dcraw -v -w -o 2 -4 -T -O p.tif SnowOnSpiderWeb.cr2

gmic s.tif p.tif bw_test

bw_test:
  l[1]
    b 3,1,1 s c nm R,G,B +/[G] [R] +/[B] [R] k[-2,-1]
    +f. 1/(1+i#0^2+i#1^2) +f. i#3*i#0^2 +f. i#3*i#1^2 rm[0,1] a c
  endl
  * s c + apply_gamma 2.2
Kovesi's functions
#2

Looks interesting, I’ll comment properly later when time is less short! For now a small suggestion for all us G’MIC posters; I often forget about the long form of the commands and how difficult the shorthand is to read (and probably quite offputting to newcomers). Where possible shall we attempt to make it as readable as we can by using long form and expanded formatting?

A while ago I did speak about auto code expansion/formatting for G’MIC but it’s a distant dream for now…

#3

As someone who isn’t a programmer, engineer or scientist, abbreviation is not the problem. I can look it up anytime using help in command line or CTRL+F on the reference page. In fact, shorthand makes the code look cleaner and more comfortable to read. E.g., G’MIC stdlib is much more comfortable to read than C++, which I don’t think I can ever learn :blush:.

Rather, the difficult part is figuring out the syntax and what everything does in layman’s terms: commands, what they do, their parameters, what their parameters mean (conceptually and value-wise), what sort of input they would expect and what sort of output they would give. The reference page, stdlib file and tutorials don’t help because they deal with the how but not so much on the what and why.

The most helpful thing that people have done for me recently is explain the darn code or command underneath the code example! Then the light bulbs turn on!

#4

Good timing :slight_smile:
I prefer abbreviation too, I’m thinking of our poor visitors! Perhaps potential users aren’t the kind who care about that anyway so who knows… as an experiment I’ll try it here anyway:

bw_test : skip ${1=10}
  to_rgb    # ensure 3 channels
  +blur $1%    # blur this as a new image
  split[1] c    # split blur channels to separate images: R, G, B
  name[-3,-2,-1] R,G,B    # rename the images
  div[G] [R]
  div[B] [R]
  remove[R] name[G,B] k,kp
  sqr[k,kp]
  +add[k,kp] name[-1] mapR add[mapR] 1 pow[mapR] -1  # mapR = 1/(1+k^2+kp^2)
  move[mapR] 1
  mul[k] [mapR]
  mul[kp] [mapR]
  append[mapR,k,kp] c    # combine channels into one image
  mul compose_channels +    # multiply original * map then = R+G+B

The author doesn’t appear to state the blur radius. I’m not familiar with ‘R’ so not certain if some parts are matrix operations rather than vectors. I’m unable to make much sense of the math scribbles either I’m afraid!
Edit: you’ll notice I skip anything regarding input from dcraw or linear gamma conversion - I leave that to the user

#5

I modified my filter slightly to match yours

bw_testv2:
  +l
    b 10% s c nm R,G,B +/[G] [R] +/[B] [R] k[-2,-1]
    +f. 1/(1+i#0^2+i#1^2) +f. i#32*i#0^2 +f. i#32*i#1^2 rm[0,1] a c
  endl
  * s c +

but the result didn’t match, until I discovered that I was using the wrong index value. I wonder why i#3 doesn’t produce an error and is equal to i#0.

(G'MIC staff) #6

Image indices in math parser are indeed cyclic, so you’ll never get an error with out-of-bounds indices.
Two reasons for that:

  • Using negative indices is often convenient : i(#-1,x,y).
  • As the indice can be a variable, e.g. i(#ind,x,y), we don’t want to check if ind is out-of-bounds because it would be slower than applying a simple modulo. And as the math parser code is often run on millions of pixels, these extra tests could make a difference on the execution time.

I wonder if the entire command you wrote could not be more easily written using only the math parser.
I’ll give a try maybe today.

(G'MIC staff) #7

Something like this maybe could do the job :

foo2 : skip ${1=10}
  repeat $! l[$>]
    to_rgb +b $1%
    100%,100%,1,1,"k2 = (G#1/R#1)^2;  kp2 = (B#1/R#1)^2;
      (R#0 + k2*G#0 + kp2*B#0)/(1 + k2 + kp2)"
    k.
  endl done

I’ve not checked the correctness of the calculations though, so maybe I’m wrong.
Anyway, the advantages are:

  • Minimize the number of image buffers allocated for the calculations.
  • Do the calculations using double instead of float.
  • Parallelized computation.
  • Shorter code :wink:

Does this look OK ?

(Jean Pierre “Xaos52” De Schacht) #8

My ‘dcraw’ does not have an -O option to specify the output file name.
Does your version have it or is this a typo?
I am on linux Debian Sid up-to-date.

#9

Looks okay. I will check later. I was following Guillermo’s code but this is what I would have worked toward :slight_smile:.

Too intentional for a typo, don’t you think? :stuck_out_tongue: Regular dcraw doesn’t have this switch.

Speaking of radius, is there a difference between std_variation[%] and radius[%]? Is one statistical and the other spatial? Is there a way to determine an optimal value when dealing with noise? I have tried a few things in the past but none of it is based on anything concrete. Excuse my ignorance :blush:. Thoughts?

#10

Yes you’re right, once again I’m loose with terms :grimacing:
It is indeed a statistical scale parameter rather than a radius. As for optimal values and noise, even basic probability theory is outside my knowledge I’m afraid (it’s on my todo list… :smile: ). Radius is for me just a convenient way to think about it, like a spatially weighted average.

#12

@David_Tschumperle Could you explain where std_variation[%] comes from? E.g., if I use %, what is it a percentage of? Standard deviation?

(G'MIC staff) #13

Yes, basically a radius or size argument in a command means you have some fixed size used to do something in that command (e.g. blur with boxfilter).
When you have a std_deviation it means the kernel used has virtually an infinite support but has values decreasing following a gaussian distribution of given std_deviation. Usually, you can consider that the equivalent radius would be something like radius = 3.5 x std_deviation.
Now, if you add a % to the argument, this means it is computed from the image width and height (and depth if any), often using something as new_std = max(width,height,depth)*std_%/100
(this may depends on what the command does of course, sometimes depth won’t be considered for instance, if the command is intended to work separately on 2d slices).

(Alan Gibson) #14

“-O” isn’t an option in dcraw as distributed by Dave Coffin. It is in distributions that come with ImageMagick, and is used to set the name of the output file. It is a useful addition, so you might also find it in other patched versions (such as my own).

My ignorance of Spanish prevents me from understanding the “El momento decisivo” article, and my ignorance of G’MIC prevents me from understanding the shorthand codes. Sure, I can look then up, but it’s like attempting to translate Spanish by looking up every word in a dictionary.

@garagecoder’s version is more understandable, for me.

G’MIC shorthand is like any jargon: useful for people who know it, and a barrier for others.

As I don’t understand the topic, I hesitate to answer.

As others say, a radius implies a binary choice: pixels within the radius are fully changed, and those outside aren’t changed at all. We often prefer a fading effect, and a Gaussian (or “bell curve”) fade is often used. For that, the usual measure is standard deviation (aka “stddev” or “SD”). See https://en.wikipedia.org/wiki/Standard_deviation

Noise in digital photos usually affects individual pixels, so I expect filter radii would be around 1 to 2 pixels, or SD around 0.3 to 0.6.

EDIT: I should add: I’ve never heard of a “standard variation”. I don’t know what that is. Perhaps a typo for “standard deviation”, or for “variance”.

#15

I wasn’t very helpful with my answer to @xaos52. If he is still following, it is possible to name the output without the -O option. I would use

-c        Write image data to standard output

and then redirect it to a file.

@snibgo Thanks for your thoughts. I liked the way you included a diff file for your custom dcraw so that we know what you changed. Also, I think 0.01% is a good clipping level, but as you mentioned, considering the other channels would be a good idea. Personally, I prefer unbounded output.

I was like that when trying to learn a second language :crazy_face:. What I meant was that, in my perspective, the long form is only slightly less cryptic than the shorthand; I have to look up the commands anyway. However, in your case, it is harder to skim and provide feedback :blush:. I liked the way @garagecoder added comments next to the code. I will try to do that next time.

Good: that is how I have been approaching the problem, although I am still unsure how SD works. E.g., where does 0.3-0.6 come from? Is this related to the 68–95–99.7 rule?

@David_Tschumperle Looks like you have changed std_variation to std_deviation. It makes sense now. Would it also be appropriate to make this adjustment in the stdlib? One followup question is where does radius = 3.5 x std_deviation come from? Is 3.5 a constant or a variable? Also, does new_std = max(width,height,depth)*std_%/100 mean “percentage of max dimension multiplied by standard deviation”?

One related question to everyone is what is the definition of radii wrt to the filter kernel? E.g., what is the width of the actual window when radius=2?

#16

This is good to know about. I know you have less time these days, but is there possibility to generate a simple delimited list of all built-in commands with the shorthand equivalent? Example:

div,/
mul,*
blur,b

If this is available there’s a greater chance somebody can make simple code formatting tools (or the reverse; compression/obfuscation!).

#17

@garagecoder G’MIC already compresses the stdlib. Maybe you could rig that part of the code to switch between long form and shorthand… Perhaps even beautify the code along the way :wink:.

(G'MIC staff) #18

Yes, indeed, that is a typo I’ve made multiple times.

For gaussian functions, it is often assumed that when |x|>3.5*std_deviation, then the function value is close enough to zero.

No, it means : max dimension * std_deviation (expressed in %) / 100.

#19

Hmm maybe I’m wrong but I think at various points it’s either whitespace removal or a usual data compression algorithm, rather than translation to/from short forms.
@David_Tschumperle is my earlier proposition possible somehow?

#20

Would it produce a 5x5 window? Or something else?

image(Image Source)

#21

Most commands I think now use interpolated input, with type sometimes specified as a parameter. Not sure of the type used for this (maybe just linear?), but even convolution commands such as ‘convolve’ will be exactly centred and use interpolated input. In older versions this was not the case.