Automatically detect - remove shapes on more images (and inpaint them)

@David_Tschumperle

Bonjour,

It’s a good idea to create this function to detect circles.
I saw that this function existed in the OpenCV library : HoughCircles()
In the examples found on the internet this function is very simple to use.
I await the equivalent in G’MIC with great interest :o)

Hello @samj

I saw that this function existed in the OpenCV library : HoughCircles()

A colleague of mine, told me it could perhaps be available within R as well (https://www.r-project.org/).

In the long past, 10 years ago or so, at work, we had some petri dish where we need to measure the area of the fungi growing on them.
A researcher wrote a script to detect them from my jpeg pictures and It worked extremely well. Unfortunately, he doesn’t work with us anymore :slight_smile:

CHT - Was reading up on it in the wee hours. Alas! @Silvio_Grosso has shot — what, ten million? — botanicals already with five euro pieces; otherwise suggesting using a square target was on my lips, and the present hough would handle squares in a straightforward way.

Remarks on using matchpatch to follow. Not a total failure, but not 100% success either.

Indeed, matchpatch won’t manage changes in color/intensity/sizes between the actual photograph and the template image very well.
I don’t think it’s a good approach considering the variety of photographs shot by @Silvio_Grosso .

Been a fun evening. Thank you, @Silvio_Grosso !

apple1_out
That was pretty slick.
apple2_out
OOPS! Coin too close to the apple. Can’t do anything about that. Also, had to tweak settings a bit to have success. In the real world, that means manual intervention.
blueberry_out
Ah — this was actually a FAIL, because I had to use a lightened version of the “standard euro coin”. In the real world, that also means intervention.
coinplant_out
Success. This, of course, was the sweet spot photograph where all the stars align.

This is my script, for the curiosity of it all

matchit : -check ${"is_image_arg\ $1"} -skip ${2=0.25},${3=5},${4=0.25}
   cutoff,ed,ps=${2-4}
   pass$1 1
   =>. patch
   foreach[^patch] {
      ma=-1
      ml=-1
      bc=-1,-1
      =>. unkwn
      pass[-1] 1
      =>. patch
      +matchpatch[unkwn] [patch],{$ps*w#$patch},{$ps*h#$patch},1,20,0,-0.005,1   d ,
      -channels. 2                                                             d ,
      -fill. ">begin(cv=$cutoff*(iM-im);print(cv));i(x,y)<cv?1:0"              d ,
      -erode. $ed
      -dilate. $ed
      -label_fg. 0,1                                                           d ,
      =>. areas
      -repeat {iM#$areas}
         [areas]                             
	 -eq. {$>+1}
	 +barycenter.
	 -area.. 0,1
	 -if i(#-2,{@0,1})>$ma
	    ma={i(#-2,{@0,1})}
	    ml={$>+1}
	    bc={@0,1}
	 -fi
         -keep[areas,unkwn,patch]
      -done
      -fill[areas] i(x,y)==$ml
      +autocrop[areas]
      -circle.. $bc,{1.5*sqrt(w^2+h^2)/2},1,1
      -remove.
      -inpaint_morpho[unkwn] [areas]
      -remove[patch]
   }
   -remove[patch,areas]

And these were the command lines that produced the images above.

 gmic matchit.gmic -input coinplant.jpeg -input coin.png matchit[-2] [-1],0.01,5,0.16   OK
 gmic matchit.gmic -input apple1.jpg     -input coin.png matchit[-2] [-1],0.01,5,0.16   OK
 gmic matchit.gmic -input apple2.jpg     -input coin.png matchit[-2] [-1],0.03,5,0.20   ADJ Coin too close to botanical
 gmic matchit.gmic -input blueberry.jpg  -input coin_light.png matchit[-2] [-1],0.05,5,0.35 ADJ+ADJMSK

So — if I had to intervene to adjust arguments or change the matching patch, I marked it as a ‘FAIL’. So I’m batting about 50% — way too low for the Real World. I’d like to be able to load a hundred images and realize success in about 98% of them. I could tolerate fixing two out of a hundred. But not half. May as well use GIMP. As David could attest, matchpatch plays a statistical game, but matchpatch can’t tolerate modest levels of variance. Loosen restrictions, and bit and pieces of plants look too much like euros. Tighten restrictions, and the euro is never seen. I started looking at CHT at about 11 PM, then dozed off around midnight. I’ll be fine after about 20 ounces of strong coffee…

Onto CHT!!!

2 Likes

Hello @grosgood

@Silvio_Grosso has shot — what, ten million?

Milion not, but thousands is quite likely :slight_smile:

Just to give you an example of a “single” session of a working day:

I always try hard to take only one very best shot per sample in order to reduce the amount of fixing up later (with RawTherapee and GIMP). For instance, I always use a tripod, the lowest amount of ISO etc etc

A former colleague of mine, now retired, told me that “a good photographer must try to learn how to shoot the least amount of pictures per day”.
Usually beginners take a lot of pictures because they are not sure about the final results (and usually have plenty of time to fix them later on…).

Perhaps a simple general rule will cover 98% of cases. For example, “The coin is the smallest circle that is not inside a larger circle, and with a radius that is at least 15 pixels”.

And perhaps another simple rule will cover 98% of the rest.

HA! Economy in expression! How often that crops up in so many endeavours!

The Writer’s Lament: “I am sorry I wrote such a long letter. I did not have time to write a short one.”

Attributed, I believe, to Sun Tzu: “To gain knowledge: add. To gain wisdom: subtract.”

As @David_Tschumperle noted, the better euro coin finder is probably the Circular Hough Transform, a generalization of the linear version already in G’MIC. The major pieces for a generalization are lying around. BUT, even there I foresee difficulties: round petri dishes, round bacterial colonies, round euros. Betwixt Scylla and Charybdis there is but a narrow way…

EDIT:

There is that…

Previous pictures of petri dish were about fungi.

As regards the bacteria, here they are:

For bacteria, which are usually white-yellowish, I usually put a black background.

Erwinia amylovora, in this picture is quite a big problem for plants in the USA too.
Actually, it was “imported”, by accident, in Europe from the USA many years ago and now it is quite established :slight_smile:

That’s why it is good that you are the plant pathologist. me: “bacteria, fungi, whydayawannaknowthediff anyway?”

You probably all found this already but here is a cod e for opencv in C++, java and python :slight_smile:
tutorial_hough_circle

1 Like

Not to forget Digital Image Processing with C++ written by some guy we see around here and his buddies. Section 6.2, page 128. Hough transform. Go buy the book; the authors have families to feed…

Hahaha so do i… So if the authors agree for a refund i will buy it :wink:

Considering the very low royalties earned on each sale, I’dreally need to sell a lot to be able to support myself!

Perhaps @Silvio_Grosso can help. He seems to have a lot of 0.05 euro coins handy…

…Or maybe just one though.

Hello everyone,

First off, thanks a lot indeed to anyone for your suggestions and input.
In all truth, I have opened this thread mostly out of curiosity because for very important images (e.g. the ones which I e-mail to someone else) I delete the coins manually through GIMP and G’MIC.

Within the mycological group of which I am a member I take several pictures as well.
They are for pleasure. No real work involved :slight_smile:
Over the years, I have adopted a different “setup”: green background and coin of 1 euro :slight_smile:

At present, with G’MIC I am mainly worried about the loss of my Exif data (with the command line interface).
For my personal needs, at work, this is a real deal breaker :frowning:
From what I have gathered, over the years, there is no way to avoid that, right?

I don’t know much about programming (only a tiny bit of SQL) therefore I don’t know whether this loss occurs also with other frameworks (e.g. ImageMagick)

With GIMP and G’MIC (through the GUIs) this data loss does not occur.
All the Exif data are preserved after the G’MIC filter has been applied.

Sadly, yes, ImageMagick doesn’t copy all the exif metadata. Exiftool can be used for that.

Conic Hough

Careful - one typo away from Comic Hough — which epitomizes various aspects of this project.

  1. Armillaria mellea reposing on green with its five euro piece.
  2. Same. Flattened to near graphic shapes. Conic Hough locates the euro piece and masks it in orange.
  3. An image containing a column vector. The script consumes its selected images and replaces them with column vectors of likely circle centers. Here, the column vector has one entry with three components: [x,y,r]; the first two components are the coordinates of where Conic Hough thinks the center of the five euro piece resides. r is the distinguishing radius of all the circles that Conic Hough aims to seek out and mark.
$ gmic chough.gmic -input Armillaria_mellea.jpg v + chdemo. 83,0.2,2,1

chdemo: Toy framework to preprocess images for consumption by chough, which implements the Conic Hough algorithm. Preprocess ⇒ means ‘Flatten to near-graphic shapes.’ Hit the arrow for the chough.gmic code for details. Some of it is even commented.

chough.gmic
#@cli chough : radius>0,_sensitivity>0,_precision,_verbose
#@cli : Select: image(s) to be scanned for circular-like artifacts
#@cli : of pixel 'radius'. Execute Conic Hough. Locate the 
#@cli : noisy clusters and compute median points of same. Replace
#@cli : selected with a 1xN vector array list of N discovered median
#@cli : points. 2D coordinates + given radius parameter for downstream
#@cli : plotters to draw circular masks centered on the median points.
#@cli : As 'sensitivity' approaches zero, more suspected artifacts
#@cli : could turn up, but at the price of longer analysis. Could
#@cli : also turn up just noise instead of artifacts. Increasing
#@cli : 'precision' sizes an internal 'parameter space' image.
#@cli : Argument is a multiplier that increases the width and height
#@cli : of the selected input image. Factors of 2-4 is less susceptible
#@cli : noise, but consumes more memory.
#@cli :
chough : check "${1}>0 && ${2=0.01}>0 && ${2}<=1 && ${3=2}>=1.0 && isbool(${4=0})"
   rad,sen,pr,verb=${1-4}
   -foreach {
      nm={-1,n}
      -name. inputimage
      ow={w#$inputimage}
      oh={h#$inputimage}

      # Sprite diameter
      sd={$pr*$rad+1}
      -luminance[inputimage]
      -gradient_norm[inputimage]
      -fill[inputimage] ">abs(
                             i-j(1,1,0,0,0,1)
                             )>$sen*iv?
                       1:
                       abs(
                             i-j(1,0,0,0,0,1)
                          )>$sen*iv?
                       1:0"

      # Try to image process circular-like artifacts
      # as thin, single-pixel circles.
      -closing_circ[inputimage] 3
      -if $verb
         -display ,
      -fi
      # Performance killer. More than half of runtime
      # takes place in thinning...
      -thinning[inputimage] 1
      -if $verb
         -display ,
      -fi
      # Draw circular sprite centered on suspected
      # rims of suspected circular artifacts. Also
      # drawing on found noise. Point cloud is non-
      # background-colored pixels. Could be anything,
      # maybe even five-euro coinage.
    
      -pointcloud[inputimage] 3
      -if w#$inputimage
         -channels[inputimage] 0,1
         -fill[inputimage] "$pr*I"                    
         -input {$ow*$pr},{$oh*$pr},1,1
         -name. paramspace
         -input {2*$sd},{2*$sd},1,1
         -name. sprite
   
         # Faster to draw just one circle, then rubber-
         # stamp it with image command.
   
         stat={ellipse(#$sprite,$rad*$pr,$rad*$pr,-$sd,-$sd,0,1,0xffffffff,255)}
         -repeat {w#$inputimage}
            cx,cy={I(#$inputimage,$>,0)}
            -image[paramspace] [sprite],{$cx-$rad*$pr},{$cy-$rad*$pr},0,0,0.002
         -done
     
         -keep[paramspace]
         -normalize_local[paramspace] 10,200,4%,4%,1,0,255
         -if $verb
            -display ,
         -fi
         -fill[paramspace] 'i>=0.97*iM?1:0'
         -if im==iM
            -echo[^-1] "No\ detected\ coins\ @\ "$rad"\ radius."
         -else        
            # Maybe coins? Got clusters of crossing conics...
            # Find mean points of the noisy patches.
            -_medianofclusters[paramspace] 32,$verb
            +fill[paramspace] "V=I(x,y);V=V/$pr;V[2]=$rad;V"
               -keep.
               -name. $nm
         -fi
      -else # Seems that w#$inputimage == 0 ... 
         echo[^-1] $nm"\ seems to have no features. No points found."
      -fi
    }

_medianofclusters: -skip ${1=32},${2=0}
   ed={$1^2}
   verb=$2
   -foreach {
      -pointcloud. 3
      -channels. 0,2
      -if w>1
         # Consolidate clusters: find
         # their average centers.
         nm={-1,n}
         -name. cloud
         -input 0
         -name. cstack
         -shift[cloud] 0,0,0,1,2
         -permute[cloud] cxzy
         -eval ">P=crop(#$cloud);
                 psz=size(P)/3;
                 erb=$ed;
                 vrb=$verb;
                 cc=0;
                 ACC=vector2(0);
                 if(vrb,print(P));
                 for(
# --------------- INIT                       
                     t=1;
                     k=1;
                     Q=P[1,2];
                     ACC+=Q;
                     if(vrb, print(ACC));
                     chk=P[0];
                     P[0]=t;
                     cc=1,
# --------------- CONDITION                       
                     chk==0,
# --------------- PROCEDURE                       
                     if(
                          chk==0,
                          t=t+1;
                          k=1;
                          P[0]=t;
                          cc=1;
                          Q=P[1,2];
                          ACC=Q
                       ),
# --------------- BODY                       
                     do(
                          if(vrb, 
                               print(Q);
                               print(P[3*k,3])
			    );
                          if(
                               P[3*k]==0,
                               err = mse(Q,P[3*k+1,2]);
                               if(vrb, print(err,erb));
                               if(
                                   err<erb,
                                   P[3*k]=t;
                                   cc+=1;
                                   ACC+=P[3*k+1,2];
				   if(vrb,
                                        print(P[3*k,3]);
                                        print(ACC);
                                        print(cc)
				     )
                                 ),
                               _(DO BREAK: Remaining points in P already clustered);
                               break()
                             );
                          k=k+1;
                          if(vrb, print(k)),
                          k<psz
                       ); _(Do clustering);
                       da_push(#$cstack,[ACC/cc,t]);
		       _(P sort: Shuffle unclustered points to front);
                       P=sort(P,1,psz,3);
                       chk=P[0];
		       if(vrb,
                            print(t);
                            print(chk);
                            print(P)
			 )
# --------------- END FOR FINDING CLUSTER CENTERS                       
                    ); _(for: Find median points of clusters);
                 if(vrb, print(P));
                 da_freeze(#$cstack)
               " # eval math expression argument
         -remove[cloud]
         -name. $nm
      -else
         -echo[^-1] "Given image seems to have no circular-like artifacts. Nothing done."
      -fi
   } # foreach image in command's selection
   
#@cli chdemo: radius>0,_sensitivity>0,_precision,_verbose,_sensitivity>0,_precision,_verbose
#@cli : Toy framework
#@cli :
chdemo : check "${1}>0 && ${2=0.01}>0 && ${2}<=1 && ${3=2}>=1.0 && isbool(${4=0})"
    rad,sen,pr,verb=${1-4}
    -foreach {
       +median. 40,80
       -repeat 3
          -smooth. 80,0.8,0.3,0.8,1.1,0.8,7
       -done
       -if $verb -v + -fi
       +chough. $rad,$sen,$pr,$verb
       -if $verb
           -display ,
       -fi
       -eval. "
                vrb=$verb;
                PP=crop(#-1);
		if(vrb, print(PP));
                repeat(
                        size(PP)/3,idx,
                        ellipse(#-2,PP[idx,2],PP[idx+2],PP[idx+2],0,1,[255,170,0])
                      )
              "
    }

Arguments:

  1. Radius (83 pixels). Conic Hough won’t notice circles of other radii. Nor ellipses. Nor messy piles of mushroom roots. The algorithm is picky on this. ± one or two pixels and the euro is passed over as Not A Matching Circle. More on that below.
  2. Sensitivity. (0.2 — Bigger ⇒ marginal imagery is more likely to not even be considered) Too big (0.4+) and euros may not be noticed. Too small (0.05–) and too much is noticed.
  3. Parameter space multiplier (2). Internally, Conic Hough plots circular ‘sprites’ in a ‘parameter space.’ This is a a grayscale, multiplied up from the given image. Conic Hough draws some ≈42,000 sprites, very — very! — faint sprites, on this parameter image. This tracing process into parameter space works off the edge image, which, in turn, has been derived from the source image, and which Conic Hough follows, transferring the path to parameter space and tracing with sprites (faintly!) into that parameter space. Should there be an edge image that looks like a circle — exactly with a radius of 83 pixels — then stars align and the tracings of an 83 pixel circular sprite (very faint!) around the circumference of an 83 pixel radius, circular edge image, transferred to parameter space, causes a Hotspot! to appear exactly in the center of the edge image. That’s where the rims of all the faint circles criss-cross and criss-cross and criss-cross, since the radii match. Then it is just a matter of locating the coordinates of the hot spot. It corresponds to the center of the five euro piece, which has a radius of ≈83 pixels in the image. This hot spot won’t happen if the edge image is larger or smaller than the sprite. See the graphics following. This tracing of matching circles, sprite and edge image, is the gist of the Conic Hough algorithm’s workings.
  4. A boolean flag. Turn it on and Conic Hough dumps bits and pieces of its internal state to gmic’s log, and stops here and there to display intermediary parameter space images. See the code listings above.

Parameter space gallery
Some internal images illustrating the progression of Conic Hough:

If a photograph can be coaxed into a state of Ligne claire — perhaps like a frame from Hergé’s The Adventures of Tintin, then there is a hope for finding the five euro piece. If the coin slips from the depth of field and is blurry, then Conic Hough just can’t cope: it needs edges to position sprite plotting.

Ditto. Morphological thinning. This is the time pig. More than half the run time is in thinning, but it delivers on Ligne claire. Pondering other approaches is on the TODO.

Some forty-two thousand 83 pixel circular sprites tracing the edge lines on the parameter space image makes a hotspot! where the stars align: the tracing of 83 pixel radial sprites along the circumference of an 83 pixel radial edge image engender a pileup at the exact center of the parameter space circle image. Any other radius won’t work — like the mushroom heads. This is the essence of Conic Hough. Most everything else is book-keeping.

What’s Next?

Fix what is missing: Conic Hough in a 3D parameter space can do trial radii, say, ± 20% around a target radius. The present implementation furnishes no lee-way on the euro piece changing size from input image to input image. Coins move around. Cameras move around. This implementation will miss the euro if the radius is just one or two pixels off. That won’t work in ‘set-and-forget’ production environments.

So this isn’t production. But the way forward is clear. Hack on it some more next weekend. In the meantime, everyone have fun with what’s here.

2 Likes

Bugs in my head
Still not production; still a work-in-progress. “Progress,” this last week, has mainly entailed throwing out various pre- and post- filter conditioners: a reductionist effort with an aim of seeing what can be gotten away with — a search to find what is really essential in this task of picking out Euro coinage from among the fungi or bacteria (I can’t tell the difference). Should you care to, diff -u the following with the chough.gmic listing of Post 42.

chough.gmic - 8fb568ec 'pixlpost529'
#@cli chough : radius>0,bkgrndnoiseelim>0,_precision,_verbose
#@cli : Select: image(s) to be scanned for circular-like
#@cli : artifacts of pixel 'radius'. Execute Conic
#@cli : Hough. Locate the noisy clusters and compute median
#@cli : points of same. Replace selected with a 1xN vector
#@cli : array list of N discovered median points. 2D
#@cli : coordinates + given radius parameter for downstream
#@cli : plotters to draw circular masks centered on the median
#@cli : points. As 'bkgrndnoiseelim' increases, single-pixel
#@cli : noise decreases, but detected edges may dissolve as
#@cli : well. Increasing 'precision' sizes an internal
#@cli : 'parameter space' image. Argument is a multiplier that
#@cli : increases the width and height of the selected input
#@cli : image. Factors of 2-4 is less susceptible noise, but
#@cli : consumes more memory.
#@cli :
chough : check "${1}>0 && ${2=15}>0 && ${3=2}>=1.0 && isbool(${4=0})"
   rad,bne,pr,verb=${1-4}
   -foreach {
      nm={-1,n}
      -name. inputimage
      ow={w#$inputimage}
      oh={h#$inputimage}

      # Sprite radius
      sd={$pr*$rad+1}

      -edges[inputimage] $bne%
      -oneminus[inputimage]

      # Upcoming point cloud has substantial
      # redundancy. Trimming half has little
      # effect on solution
      -resize[inputimage] 50%,50%,1,1,1
      -resize[inputimage] 200%,200%,1,1,4
      -pointcloud. 3
      -if $verb
         -display ,
      -fi
      # Draw circular sprites centered on suspected
      # rims of suspected circular artifacts. Also
      # drawing on found noise. Point cloud is non-
      # background-colored pixels. Could be anything,
      # maybe even five-euro coinage.

      -if w#$inputimage
         -channels[inputimage] 0,1
         -fill[inputimage] "$pr*I"
         ox,oy={I(#$inputimage,0,0)}

         # Shift sprite image by way of deltas
	 # from point-to-point. Get those deltas
	 # by subtracting the pointcloud from a
	 # one-plot-point-shifted copy of itself,
	 # i.e., foreach p(n)-p(n-1)

         +shift[inputimage] -1,0,0,0,2,0
         -reverse[-2,-1]
         -sub[-2,-1]
         -name. diffs
         -input {$ow*$pr},{$oh*$pr},1,1
         -name. paramspace
         -input [-1]
         -name. sprite

         # Draw just one circle, then rubber-
         # stamp it with shift/add commands.

         stat={ellipse(#$sprite,{w/2},{h/2},-$sd,-$sd,0,1,0xffffffff,1.0);1.0}
         -shift[sprite] {$ox-w/2},{$oy-h/2},0,0,2,0
	 -if $verb
	    -display ,
	 -fi
         -repeat {w#$diffs}
            -add[paramspace] [sprite]
            -shift[sprite] {I(#$diffs,$>,0)},0,0,2,0
         -done
         -keep[paramspace]
         -if $verb
            -display ,
         -fi
         -fill[paramspace] 'i>=0.97*iM?1:0'
         -if im==iM
            -echo[^-1] "No\ detected\ coins\ @\ "$rad"\ radius."
         -else
            # Maybe coins? Got clusters of crossing conics...
            # Find mean points of the noisy patches.
            -medianofclusters[paramspace] 32,$verb
            +fill[paramspace] "V=I(x,y);V=V/$pr;V[2]=$rad;V"
               -keep.
               -name. $nm
         -fi
      -else # Seems that w#$inputimage == 0 ...
         echo[^-1] $nm"\ seems to have no features. No points found."
      -fi
    }

#@cli medianofclusters: errorball>0,_verbose(bool)=False
#@cli : Find clusters in pointclouds; compute their
#@cli : medians. Package these in a column vector of
#@cli : centers.  errorball: defines cluster radius. Points
#@cli : farther apart than this distance are not in the
#@cli : same cluster.  verbose: When true, emits debug
#@cli : data to G'MIC shell log.
#@cli :
medianofclusters: -skip ${1=32},${2=0}
   ed={$1^2}
   verb=$2
   -foreach {
      -pointcloud. 3
      -channels. 0,2
      -if w>1
         # Consolidate clusters: find
         # their average centers.
         nm={-1,n}
         -name. cloud
         -input 0
         -name. cstack
         -shift[cloud] 0,0,0,1,2
         -permute[cloud] cxzy
         -eval ">P=crop(#$cloud);
                 psz=size(P)/3;
                 erb=$ed;
                 vrb=$verb;
                 cc=0;
                 ACC=vector2(0);
                 if(vrb,print(P));
                 for(
# --------------- INIT
                     t=1;
                     k=1;
                     Q=P[1,2];
                     ACC+=Q;
                     if(vrb, print(ACC));
                     chk=P[0];
                     P[0]=t;
                     cc=1,
# --------------- CONDITION
                     chk==0,
# --------------- PROCEDURE
                     if(
                          chk==0,
                          t=t+1;
                          k=1;
                          P[0]=t;
                          cc=1;
                          Q=P[1,2];
                          ACC=Q
                       ),
# --------------- BODY
                     do(
                          if(vrb,
                               print(Q);
                               print(P[3*k,3])
                            );
                          if(
                               P[3*k]==0,
                               err = mse(Q,P[3*k+1,2]);
                               if(vrb, print(err,erb));
                               if(
                                   err<erb,
                                   P[3*k]=t;
                                   cc+=1;
                                   ACC+=P[3*k+1,2];
                                   if(vrb,
                                        print(P[3*k,3]);
                                        print(ACC);
                                        print(cc)
                                     )
                                 ),
                               _(DO BREAK: Remaining points in P already clustered);
                               break()
                             );
                          k=k+1;
                          if(vrb, print(k)),
                          k<psz
                       ); _(Do clustering);
                       da_push(#$cstack,[ACC/cc,t]);
                       _(P sort: Shuffle unclustered points to front);
                       P=sort(P,1,psz,3);
                       chk=P[0];
                       if(vrb,
                            print(t);
                            print(chk);
                            print(P)
                         )
# --------------- END FOR FINDING CLUSTER CENTERS
                    ); _(for: Find median points of clusters);
                 if(vrb, print(P));
                 da_freeze(#$cstack)
               " # eval math expression argument
         -remove[cloud]
         -name. $nm
      -else
         -echo[^-1] "Given image seems to have no circular-like artifacts. Nothing done."
      -fi
   } # foreach image in command's selection

#@cli chdemo: radius>0,_rblur>0,thres,_precision,_verbose,_sensitivity>0,_precision,_verbose
#@cli : Toy framework
#@cli :
chdemo : check "${1}>0 && ${2=2}>0 && ${3=15}>0 && ${4=50}>0 && ${5=60}>0 && ${6=25%} && isbool(${7=0})"
    rad,hc,bne,rblur,thres,pr,verb=${1-7}
    -foreach {
       nm={-1,n}
       ow={w#-1}
       oh={h#-1}
       sc={s=['$pr'];s[size(s)-1]==_'%'?(s2v(s))/100.0:s2v(s)}
       -if $sc>1
          -error "Downsampling "$pr" should be a percentage < 100% or decimal fraction < 1.0"
       -fi
       +resize. {$sc*w},{$sc*h},{d},{s},5
       -median. $rblur,$thres
       -if $verb -v + -fi
       +chough. {$rad*$sc},$bne,1,$verb
       -fill.. 0
       -resize.. {w#-2/$sc},{h#-2/$sc},{d#-2},1
       -resize... {w#-3/$sc},{h#-3/$sc},{d#-3},{s#-3}
       -repeat {h#-1}
          cx,cy,r={I(#-1,0,$>)/$sc}
	  -ellipse.. $cx,$cy,{$r+$hc},{$r+$hc},0,1,255
       -done
       -inpaint_matchpatch... [-2]
       -keep...
   }

What follows goes under the rubric of “ancedotal asides”: of no use toward reaching the eventual form that this filter should take. If you want, leave the theatre now and get out to the parking lot before the main onrush.

As you may have gathered, these various Hough tools rely upon some voting scheme, that if a feature is present in an image, such as a euro coin with an 80 pixel radius, some scheme will upvote a particular pixel in a “parameter space” image — this scheme will upvote it many, many times, and that some aspects of this upvoted pixel will tell us something about the feature. In the present case, the pixel’s location in parameter space exactly coincides with the euro coin’s location in the original image. That is because we have so declared the parameter space to be in a one-to-one, pixel-to-pixel mapping with the original image, and that our design of the voting scheme is to adhere to this one-to-one mapping. So parameter space voting is important (“Vote early; vote often.” as we say in Brooklyn.); we exploit the accumulation of votes in parameter space to find euros in the original space.

The voting scheme in play here consists of:

  1. a circle drawn in an off-to-one side “rubber stamp” image — let us call it a “sprite”. This sprite comes to us by way of the ellipse() math function. Now, having drawn it, let us put the sprite to one side and come back to it later.
  2. We take all pixels from the original image that seem to be sitting on edges — harnessing a heavily edge-detected version of the original — and make a pointcloud from it: its first argument chooses one behavior from a possible four. We pick option ‘3’ harvest a pointcloud from an image; the other options entail plotting a point cloud onto an image in various ways.
  3. This brings us to the inner loop. Trusting that the elements in the pointcloud collection all locate pixels sitting on edges, we take each point, one by one, and hand that coordinate pair over to the image stamper.
  4. The image stamper’s lot in life is to regard the given point as the center of the sprite. So it takes the sprite, positions it in parameter space over the center, and stamps! All pixels displaced from the center point by the radius of the sprite receive an up-vote, this in the form of a slight gain in its luminosity value — an up-vote that is a “simple linear addition” of a fractional value to that which has already accumulated in the pixel from previous stampings — or so I thought.
  5. This, ladies and gentlemen, constitutes the genesis of the bug in my head: simple linear addition. Though bugs in the head may manifest themselves as defective code, such is generally invisible to the coder possessed by bugs in the head. That is how defective code generally passes the coder’s visual inspection. S/He sees the code, but reads something else, like “simple linear addition.” But the code doesn’t actually do “simple linear addition.”

Bugs in the head to one side, the alignment of stars that this voting scheme seeks take the form of those edge point locations from the point cloud that happen to form a parameter space circle exactly the same radius as that of the circular sprite. For as the sprite stamps — using such magical edge points as centers, there is one point in parameter space that gets up-voted with every stamp. That is because it is center point of the circle the magical edge points form, a circle that matches the radius of the sprite circle. The circumference of the sprite circle crosses the center point of the parameter space circle because their radii match. If it were otherwise then the sprite circumference would miss the center point stamp after stamp.

Here is the image stamper as originally (and defectively) conceived, from the Post 42 version:

   # Faster to draw just one circle, then rubber-
   # stamp it with image command.

   stat={ellipse(#$sprite,$rad*$pr,$rad*$pr,-$sd,-$sd,0,1,0xffffffff,255)}
   -repeat {w#$inputimage}
      cx,cy={I(#$inputimage,$>,0)}
      -image[paramspace] [sprite],{$cx-$rad*$pr},{$cy-$rad*$pr},0,0,0.002
   -done

In truth, I hardly gave this code a thought (Ay, and therein lies the rub!). My (quick) read on the behavior of the “image stamper” -image was this:

a0 = 0;
ai = ai–1 + ts

where t scales the contribution from the sprite to a very small value, here 0.002, and, after five hundred up-votes, the accumulated pixel value, a500 would be 1; and after five thousand up-votes, the accumulated pixel value would be 10, and after fifty thousand up-votes the accumulated pixel value would be 100… you get the gist of my thinking. The pixel where the stars align would accumulate up-votes to the order of 10 to a 100, the rest would (likely) accumulate values far below unity, and many, if not most, would accumulate no value whatsoever.

Voting was in the bag. I moved on.

And then voting wasn’t in the bag. I stopped, winking and blinking, a deer in headlights.

Sometime after finishing Post 42, I was finding in testing that the values of hotspots — pixels with their stars in alignment — were generally not at expected accumulated up-vote values. However, with remarkable consistency, they all rather fell short at 63% of their expected value. That is, after fifty thousand up-votes, the accumulated hotspot pixel values would be 60, 62, never much past 63 — and nowhere near 100. Broadly, the behavior was this:

lerpcurves

My first thought was to do nothing — nothing is easy — as hot spot detection was working well enough. I mean, things were sort of working. That sort of thinking got me through the night, but the bright dawning of a crisp new spring day came with the realization that — dammit! — we should do things right around here. So I took the time to look at -imagereally look at -image — and ask myself if I actually understood what its final argument, opacity, does. To spare you the details, eventually Light Dawned Over Marblehead: that the final argument to -image is the mixing ratio for a linear interpolation between the sprite and the parameter space image. Since it is lerping that we are doing, the actual behavior of image stamping — and up-voting — is this:

a0 = 0;
ai = ai–1 + t(s–ai–1)

which is not “simple linear addition,” but successive linear interpolations (“lerps”) at the fixed mixing ratio of t=0.002. The result of one lerp folds into the next. So, letting i run off to infinity finds a1–(1/e) ≈ 0.63212, this for a fully opaque sprite value of s = 1. It was at this juncture that I recognized that I had bugs in my head. In effect, accumulating vote values would be progressively shaved as up-votes for successful pixels increased. -image could not composite in any fashion other than through linear interpolation. It seemed that I had to abandon -image, a pity, as it was in every other respect a nice stamper, and, being a built-in, was fast.

After casting around for a bit, feeling a bit sheepish, feeling a bit dumb, I settled on using shift as the voting scheme’s sprite positioning mechanism. Also a built-in, perhaps the actual act of voting would not be very much more expensive. -shift operates with deltas, so the point-cloud had to be shifted, then differenced with itself, keeping one unsubtracted value for initializing the sprite’s first position. Put in other terms, I took to putting the sprite on one image, positioning with one absolute MOVETO then shifting with the remaining series of relative MOVETOs. The new voting scheme looks like this:

   …
   -pointcloud. 3
   -name. inputimage
   …
   -if w#$inputimage
      -channels[inputimage] 0,1
   …
      ox,oy={I(#$inputimage,0,0)} # Absolute start point

         # Shift sprite image by way of deltas
         # from point-to-point. Get those deltas
         # by subtracting the pointcloud from a
         # one-plot-point-shifted copy of itself,
         # i.e., foreach p(n)-p(n-1)

      +shift[inputimage] -1,0,0,0,2,0 
      -reverse[-2,-1]
      -sub[-2,-1]          # diff point cloud:
      -name. diffs         # makes relative movetos…
      -input {$ow*$pr},{$oh*$pr},1,1
      -name. paramspace    # stamp onto this image…
      -input [-1]
      -name. sprite        # stamp from this image…

      # Draw just one circle, then rubber-
      # stamp it with shift/add commands.

      stat={ellipse(#$sprite,{w/2},{h/2},-$sd,-$sd,0,1,0xffffffff,1.0);1.0}
         # Single absolute moveto to position sprite
      -shift[sprite] {$ox-w/2},{$oy-h/2},0,0,2,0
         
         # Start voting… Here on out
	 # we make relative moves, then stamp.
      -repeat {w#$diffs}
         -add[paramspace] [sprite]
         -shift[sprite] {I(#$diffs,$>,0)},0,0,2,0
      -done
      …

So far, so good. The voting scheme set-up mechanics are more involved, what with pointcloud differences being taken, but there is little extra performance drag in the -shift/-add inner loop. Mathematically, the G’MIC -add does just that, so there are no longer any curious attenuations taking place in the accumulation of up-votes in parameter space.

With voting irregularities laid to rest, the key feature of allowing target coins to vary in size, at least within a tolerable range, needs to be developed. That will add temporal slices to the parameter space. How many? Not sure, at present. It shall be the work of another weekend. In the meantime, have fun with the play code.

1 Like