Automatically detect - remove shapes on more images (and inpaint them)

Hello everyone,

Just a question out of sheer curiosity because I suppose it might be possible but very difficult to do with G’MIC :slight_smile:

In short, at work, as a plant pathologist, I take many images on a daily basis: they pertain to samples of plant diseases
In every single picture I put a coin (“5 cents” - 0,05 - euros as value). It is a fast way to obtain a visual scale as a reference (for the dimensions of the samples)

Later on, time permitting, I remove the coin with GIMP and G’MIC
I open with GIMP the image and also a png image (with a black 2 cm scale inside plus a transparent background)
I resize this png with GIMP in order to make as large as the coin.
I finally remove the coin with the Inpaint multi-scale filter (by painting on red the coin)

Now the question:
Is it possible with G’MIC to automatically detect this coin and remove it by applying the inpaint multi-scale filter (the coin disappears and the whitish background is applied)?
I’am referring to hundreds of jpeg images where the coin is never on the same position :slight_smile:

Needless to say, this would be only the first problem to automatically solve, because I would also need to resize automatically the png image with the 2 cm scale :slight_smile:

This smacks of the QR Code problem ( QR code readers on phones have to find them, first). Instead of a coin, a QR code sticker of standard size and — as a bonus — containing a little squib of information about what is being photographed, can be pretty useful, methinks. Exactly how to do that in G’MIC? I’m sure it’s possible (but I have biases) and think that it could be a nice weekly project to figure out exactly how. And from there, for me, at any rate, A Beginner’s Cookbook.

Of course, if you are looking for a turn-key, drop-in solution, I haven’t got one (yet). As for feature location, since you are in a controlled environment, I would be temped to shoot two images per sample: one with the samples absent (just shoot the coin/target/QR Code) and the second with the samples. A mask can be generated from the first shot — you do a good job with even, unvarying lighting. That mask, generated from the first and overlaid on the second, furnishes a pair of images for inpaint_morpho. This seems like automation that can take images in, a pair at a time, the shot without/with pair, which can do the marker elimination and inpaint pretty nearly automatically. That’s my rough solutions sketch. The bugs and gotchas are probably built in.

1 Like

Hello @grosgood

Thanks for your reply!

I would be temped to shoot two images per sample: one with the samples absent.

Yep, in the past, I have used G’mic to remove people from pictures (e.g. with monuments) with this technique

One problem arises, though :slight_smile:
As of today, I have already shot hundreds of images :slight_smile:

Joking aside, I take my shots with a Nikon D850, as raws (NEFs, in particular)
With RawTherapee I modify them just a bit (nothing special) and I batch save them as jpeg.
It is a very time-consuming task just for a single image and every single day I have many samples to photograph.

In the past, I put a graduated ruler.
It was the easiest way to get the job done.
For pictures taken from a long distance, where the coin is not up to the task (too small), I currently always do so.

Unfortunately:

  • the ruler is not very “aesthetic” to see (at least, IMHO): for instance, when the ruler is not “straight” in the picture (my bad, since I am usually in the hurry…);
  • With macro pictures, usually, due to the short distance of the sample from the lens, the ruler in “the background” would be usually blurred therefore, you could not read the measures (mm) anymore, later on.
    The coin, instead, is always pretty much “on focus” (it is always around 2 cm) and since it is a perfect circle is always “straight” (as opposed to the ruler) :slight_smile:

@snibgo I tested with Krita NDE, and it seems you want to do something like this? Opacity altered to reveal result:

image

That’s just the mask though. I think it’s doable.

Details here:

image

Unfortunately, to get better result, I think you need to select circle manually and get it all in. I’m just not sure what to do from there to automate it, but it might be doable.

Here’s something that might point to a direction which shows it’s doable:

image

It is possible to get the radius and the center with that mask. I haven’t figured that out yet, but surely it is.

hello @Reptorian

Thanks a lot.

I worked with Krita only in the long past because, for pictures, I usually prefer the pair: GIMP - G’MIC :slight_smile:
I considered G’MIC because of the possible options to automate the results on hundreds of jpegs images.
I am not a computer scientist but I use many of its filters (through the GUIs only…)

It is not an easy task though because the coin is usually on different positions of the images (top left, top right etc). Even its visual “dimension” differs in the images since when I am close to the sample the coin is also “bigger” in the image.
The background of the picture is whitish (sheet of paper) but not completely white due to the different white balance of the different images.

With very important images, where I need to delete the coin, I do all process manually.
I remove the coin (G’MIC - Inpaint) and add the 2 cm scale (through a png image with transparent background) with GIMP

matchpatch looks like it could be your friend.

$ gmic coinplant.jpeg coin.png +matchpatch[0] [1],16,16,1,10,0,0,1 channels. 2 fill. 'i(x,y)<200000?1:0' +barycenter. e '{@0,1}'

Here’s the shell log — annotated without the image dump.

[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input file 'coinplant.jpeg' at position 0 (1 image 690x441x1x3).
[gmic]-1./ Input file 'coin.png' at position 1 (1 image 64x64x1x3).
[gmic]-2./ Estimate correspondence map between image [0] and patch image [1], using 16x16x1 patches, 10 iterations, 0 randomizations and occurrence penalization 0 (score returned).
[gmic]-1./ Keep channels 2...2 of image [2].
[gmic]-3./ Fill image [2] with expression 'i(x,y)<200000?1:0'.
[gmic]-1./ Compute the barycenter vector of pixel values of image [2].
317.99722290039062,406.3502197265625
[gmic]-4./ Display images [0,1,2,3] = 'coinplant.jpeg, coin.png, coinplant_c1.jpeg, [barycenter of 'coinplant_c1']_c1'.

The last image is the barycenter data of the center of the coin expressed as a column vector image. I also echo that data in the log. Looks pretty much to be a pixel or two from the center of the coin. The penultimate image could almost serve as an inpaint mask

Try this on some of the other tens of thousands of botanic images you may have. The matchpatch parameters are a bit magical, but these particular arguments develop a very strong signal for five euro pieces.

Tell me how this turns out. This is a candidate Cookbook article for us documenteers. Have fun!

EDIT:

OK, so while we are in the market, maybe I can also interest you in a sweet little G’MIC command called autocrop? It will shrinkwrap-and-crop around that square region in the penultimate image, and, methinks, the ratios of widths compared to the mask, along with heights, will be fair dinkum estimates of how to scale the images so that the euro piece would more-or-less be the same size throughout. The two scales would probably disagree a little bit, so you have an error estimate of sorts.

3 Likes

Hi, i was just wondering… Will this work if the coin is ( maybe just slightly) rotated in the original picture? Maybe it could just recognize a round object?

I think @grosgood gave me a idea.

Here is what I have for now, and I know it’s not ideal:

rep_tool_for_silvio_grosso_detect_and_inpaint_and_insert_2_cm:

number_of_images:=$!

base642img MSB1aW50OCBsaXR0bGVfZW5kaWFuCjU2IDU1IDEgMyAjODUyMAp4nCV4V3ic5bnt1bk52UnAuKhO73006nVGM6Ppvc/803vvVaPuhg2mQ4BAErJDSCCb7CSEJITNJiExGBs32ZYsyeq9Sy7EhPh8ypmLudCj93+/b71rrXf9s7Q4t7C4sriyMj+7MDcxev3yp798pjeTifgDoWAg7IvGkn3n3v7k6tWvro1NT09O37kzO7+6vjk3t7w0v7C4vLS4sLg4e+urv/7m1ROFZCIR9vkjiWQ6E8+kYyG3O/bj/7kxeunazWtzc7MzM7Pzy/PzB3Wg4er87Ympi79+5UQyEIzGYl6XyxuMRKKxTHn4ZDnfF8i8/Jfb46PjszPzK4t3Ju4sLiwtzs3Pzy/MTd24/MlbpzPRSDKfiXgdkNFogqw2rcEazmWyg30Jd/qd8bnLV66OL68vTU8uLS4uzi/Mz0+Pj41+eCYRiYYj8YjXajYZDQaDyWyyW01WLzhn//GhXO75P4xevHh5cnFtZWF5aWlpYXF+bmbm2q9ORP3+oN/ngnQ6o8Pl8bjdTpvXY4WsFsih955643S4/91rN25eH5tdXQNli6Db1MTV98qBaNzrsEIQZLHaQ5GIz+vzuiPxkNvjttmsVmd44Nxw5Hdf3r5x6eYsQHFhfmZ6Ymz0J5loPuG22gwaPeR0eUMhn8PuDYUDyUQomogErXaj1n/ihbO9716//uWVqbn5hbW78ys3vvzjUDAeCrjVaqPZ6XIHrB6HO+i2OT3pfC5byKVz2UTaKLbmX+t/5qMb186Pzs0tLK8u3Zj49WAsFgf3t1jszpDfE7D7fMFgLF/KxNKZTDadTJcidptB5So9nxz6z/ULV+Zn5hcnZ25/ccYZSgRtVrvdazdBFkc66wynYr5wuhBMFXPxUDTti0RKUaXeMvJ88rUPV78cm5qZuXx94fVEIhOyOdxuv8/niUaixUIwW4w7ja5kf29vb6GQT0f6SumANxFRHX+2fObq/BfjU1cuX/ufTKyUdtii8XAyEfHHIqF0MtGXtZsdntC5QjTdf6IvFEunIWsq7oegp46fenv2ws2JL25d7YuXoz7InU4E4mEvZHUEe/Pxs8N2cySTPZnLFPL9A8lgOO/3xN3estn/ztCrH90ZHbs4/r69mPfYLL5UPBjxhsOpwuBIIvbMM7lErjQ8VD5xIpcfLGXLQxm/L1oMGqHB00/+fPr6rWtXhoIlb8Bh8aQKhWzfYKnQ29/rAgPoHcmFs/1JcNFkyuVMxZNhlx9yRT365wef//u1m5c/yo4MeUMub7Y/E3D7nJ54IhiI+zyuA6a5Qj4AVjDgc8eyEV8uKbeXkrbyy/0fTt688mrsbK8v7XIXT5YiQSDWYDyfTyajoUAgHIkEE1GPNxowZ4eivkJIH7AEgu7Y8fzzF26cL5aeTUZiTv/AqYw/4E+ko9FcKZfPpZMpMPNEInmg4XB/IpYIe/1+my+RdBVGBj4Y++/oC0+HU4BRxwdDTq/fF4jE4ql8JpsvFAv5bDyRSgS88f60r5CDUgMGyB8O6X2nen86/krkjcFo1BXpHeqNh0JRr9MbiSUBufIHZflANuV0pEuJaDkfTudD9qAX8rnsAydeGB2MPZeIOSLJbK5YyidzfX0Jnz9ZKBbz2UwmBygdckd7s+50KRAaihizRbc22utKnzt7IVYeAfjG44lMsVTIlo6fHUn6o8ViIZdJZ4vlkZg3UcxEY7lAIJ30hkthyOJNmYInh/7keXoEghyRaLZczqQLhWjq5OlsGFwrl88Xy/3FRDgWj6UK8Xgx44qV0wZHzGHVJ05m382cG7aHzbnBfDKV6es/UYqnz54KeHP5dDyZzydi0Vi61N/X2zsykMz0Z6BE3u+NZZx9Z36aOJ2H/FCuFHK5XNF0fmgwPfxMMVjIZPKFVCwSy5YH+8vlvmypr9TbX0z3Bi3+WMqZPv1W9EQY8jkAYdwemzsQHjpbKD9/Il7OZHoLsWA8Vegf6C3k8+VCcaCvNPLcoNsZsRuN6eM/ip8NOXweO9BdAojcXzxVKj1zKtmby+ZS4XhpuL9YKPWW8n25/MBAaei5k6lo0CTXZMqvJV+MOx2OSCKYSIciiUB2KFs4dzyayudTkXjvyHA5mwYUyGVSub7BcqHw7ImEPeK3xnIvpX6YtpmtwUQ4AeCOB4oDifxTg6FwoZSIARkVSuVSLpMr5sE8C33l3PDpnCWd80dyL0ffyEMWpx9QNBWNBsPF/kzvMHCiQj6eGRoplQZGRgYALsWB/myqODiQOnnS7vD4okM/DvywCDlDxb6o1+sLB5OZUt9IJjDUW0xFciPD4ITZfKkErlo+OZjPF/K5k8+65Bpf6OlflE4/aUjHhkbi3mTUHsr09paHCvFMOZ/3D50FFwpHEsWBYl8Z0CLfWwR/fjqmkDm9J34/MPScLelMn+j3e31Ob75/qJwOR/LFZCxyaig/mI8lk/neTLaQLIJRZNL51Jm0UR2MDnxypvCKM+D0ZotRjy8U6u8bKEYCyWIq5Y891VcCtgR8Y7BYSCfKfflcKlOIH8+aTLH0wJUf5V70W+NgEZRy6XxmIJVIRuLZZDDvC5wZGOhP5wZKpcFCLhnv600mU8VCYjANueKJp+f/WHh60OpPB9MD5UIpmfZ6I2BBB/xJt+/UQH8ume3LZ4tJMKNcPpUGFI+X0+5oIv7zra/OnH4tqLS5YjF/NOENekPxaCQcSIU98TODffl0ujcPbKNYiqTyxVLQCgVTMX8iVvxs9+Z7p18vypR6q83q9jnADkymoqFYNpBIHT8xkC2kC1mwkPry0WyhkPNaTX6/3x2K/WB2++YXp88ct0I6gyvgdfvC8XAE3C4e9Zci+SeHowOZbAK0T0RCxXwELCpPzO1xeVN/2N36y4WfZH6YNehV3qAz6g97wslwKBqMRr2O+PGThV5A6mzYHwgEI+lkOJryaEziQGBgdXfr4tj/nn7hJYXMZoASIa85Hg1lw3Z/yuuI9J08eSKTK5XLxUwCWGI6HgwE/U6xxZ3+872drdHR0bdO/iCl0eqtPrs/mgzE+iK2iMcViCcS+eFSMZfKlcuFVCYNtB/0BxyiiOeZ1X9tb976/Oanr5x5KSnV26HowWLJJGLxoCkY99j98XjfQG8qVSyBNZ2OxxIRn93u9pa+2N/b2pi8eHPss1fOnvEqVJEhv9qRjPniSbc34oc82bQrUR7u7wXt+vpLyUjQZTZ5U/73v9mf31iduXX9+u3fDA0Pm1UxYL9RpzsbtYdjQbcrEA25gum+cm9/f19/OR/1WA2aoC/y8saDlfW11dnx65fGP3/n5HAhYFV7BlM2byFgTvhd7qDfbvcFwW4Jx7O9sZDXaTFa7Jnk83ceTi/eXVudGx2/PXFr9Ef5p4Z05rDRn0s4XGG3w2z1xaPhMFhnIDY53B47ZDVbvMnw05OPZmZ2t1fXluenJ2+PX7/wk+FzL6dF1njcH445De4kAD2SDdjttoP4ZDMotN4wEHb68j+3d3bXttdWVxbnpqenbn117ZMXRp467fI7TK6AzxZwqY0OhxcKu00GM2TUmBw2VzKZPbPw6B/L2/fXFtdWV1eWQB784trt6x+cG3m2Pxd16C3+SNpvj+dCPqfFZLE5HBaTP5wsFeJvP/r23sL0xt7mxir4rCwvLo1Ojl69/enrZ08NjQzFnJDO6/JnUy6bxwGZTBa7N9w/0O8Pf/bo0c7y4trG+sb66uoaOOrS0uadyclbE1d+/cZIcWCwPxcEocntgOwhgyuWDPnifcVi8Pm9R6sr95bnV9a2NkAcB+Ury0vri7PTt2/dGr32X2fLORBY4sFoJJFN5QvxWH5kuJQtvL346P7S8uoKiMbLW1vzK8urGxtrq8tL20vzG9NXJubH//6bN06k06d7+3r7B3MHco3ZnIM/Wfvm0d7K1s7C6ubWxsbG8vzy8sEN1zYW1tZX1nc2V+cnL/zm+azXoZHweFx2F5srlusjZ351aeLO9NTU3PIq+P/1DTC7xcW1tbUVcMe19a2tlYmr//vOk1GzWq2XCnokMplUJhQLuFyO0PDin6emb9y8PQHy8fLy0grIyQd1qysbuxsLs1N/+Wl/UC3oEUtlEiFfKJFIpHKVWqeTCJRcy9N/nRy7dWdudgFQZRlAAo4JnrC8ujh1+ffnIkoxX6JSiXlCHruzq4vN4fH4MrVcYVZLBdYXv5q/PT42uQTwAJ1WDga4NDtx+/p/FXVSmVgkFnI7Ozjsro6OLnaXRMDhCnuEWqvLLJZmfzc7fuPm1ALotPbvAS4vTE1+/mZcIxYrpUJue0trJ69HKBQI+N3szuamtrZ2Llvsjmq5kXfHJm7fGptZWDpAcxVQe+z8D+w9Uo1UyOvmcNgARqlUJBQJe7giPpvb3dUlEEukRrdV+vZfxsdv3J6bX14Bl5ubvn31JZNQKu3p6e5oA734fKGQ39XFB7gYwYuBUafulqskCqtVZXnxwvjNyflFgOra+PzY9R96pWqpRNza1MkW9Ahk3UKeWCniiaR68MJjNuqMJoOpk9quSulCb168MzY1DSa5dOerW++51QqVgCfkcvgCuUQo7RaLZQoABqTS6fUGnUYHKfm8VkanNix2vrR1c3x+cWX9zuTlP+W6JQphRxuHK+3pYHfxdAa+Qq+Sqkw2qdZiVEkVOr5cblXWNbfaw9KB99auTc0tLt+4MjoEHi7u6ASQyyUilVJhMUtMkJrXKTQ4rVar2WzSiSBIKxFqFHRHyJT8fPnanbmx65fe1fKNanaHRC5S6UAXhVSnVNsN3R1csTJpVugcfrtUplZ3cowGRXurD/I9N3VtYubizc8DSq2E39GjU/QolGLAZanFoIp6OJ1KgzFo1JtNDqdGIjOJejQSBdTKH7H0/Xp2fOrCtbdEkIHT2SnRq0QSsVyuA5JVqlIpk9Zs83lsfr/R5AIy9hjEYq1T1M62OwMv3R6bvPRZXO0Q8brYMuA/WocbvCXaIS5PD4rMCpNTY7CoNFqexKTVyvlyXrdG3hKFev98a+LzdzRmE1fA4elsBkmPkCtQa6RSjUQkAJ8enlTIB/KQiLhyg1JmNzcLndp2cxp65eroF8fFDhVXxek2+awKqQRoTmOx6PRq2YGMwBOUAqFS0qFzKsR2VZeKp5BypU5d+uOrHzs0rh6BlCOw+w1iiVhjUKnMNrPZoFWr1Brw0WrkIrHcplZqFVK5UiDTqLr1LttbX/1MHbCy5V08jcch4wnFIolCpdZbdFq9Qa/TqtUanUYiVDt1QrORrfd1ciUKWWOPz3z6i1OKtIon40gtTotaJlMIuAK5UqPX/LtOo5YYdTyu3qpR2EwynUXOlYnYYl6X1ZP9NK7086VCiVpvMFnNWqPdrhaKNGazUQ+aafVmg6xHaTPydZBY6pa3GqyCZpW9Wx0J/x6yQW1iiVyp0pmtFqPVH/NpxUrIbDpgswnyq0UayKBUGsVinUYgMcvZXVJ9u8jjelsZh5q4aqlcb7Ua9BZIpQ9FTIqDYxpNJshm1YLrgkeqVBY9Tw5pWnk6AadF6TK9YgxDnfJuk9us1RkczpBVqQwGpGIL2EE6IAWFXKG1gF1mdNlUWqehS22RyPRGLmQ/ofWq2iQ8k1XG5/OVesjjVDsTVrnVZLRAeoVMobc67VarTWuCzBYbpLdKuuQ6PVdtPa5yCdvEQqGM3yPg9EjknojBkg5qnEajHVKIlSqD1WY26PUWg9FmNbuTDj5fw29rUZmGNAFBl1jaLRIoNSKRVGwOWCyJgM5uNpk1UqXFaTUe/Cyih3R6m81kj/u0KkUXs0GnKhvSgCtihUaq0csUGpHeZYTiXrneYtHIlGaX06IF0wDYqnSQ3WIwxLzqbrWUrRLljb1AkQKpRq4xKFVqgdmlsSScIoXJogJqcOiNZiOoNOjBPHVmk8YRMHaaILFSmNeU9e3dEjHAW6dUgnM6TQ6fXq02GZUah8tktDqdwCgs4HZqBZCw3BPoBhtHrR6U9xrbhVrIoRQKRXKpXGd1BYwiO2TWyHQup16v1eiMRq1SavFaAYFUak+KX98qV1uftPu8DTqtx6fkqWVAZTab3WPRGK0GQ48jYNLpgRYMVoPFqrWYtDqNXqmJ69uaAIDPeO3Rdo3AGLALe3r4IrPTY9fLlRYz0J/PprXqpAqFzqQGdzeCh8hlKmXEwm6VKeVvpk3pLrFAbIbkfJFM7nS4IIVEa9VohLIwZHBrVAad0arXqRUWSKtRqLQyn6WrTa40/+lJY5LfpVHLtBaD3mx06TQahdqoExsFoqANgKGzmUx2g04ph8xKpcpoVLoMnWypOHPzZ1DI1i6GlEaX1WLV6YUCuVyjBfjyBAGbVafUQnqtUQnGq9Or1DIgG5uOK5EKX1n7OO7LC5lAgCqxUiOUCoHvKuQSnVygCDshMBCzXmcCkEpUepNF3MmWaZU9CrHuTzvnn/UVjDSwKbnsHhFXogSup5SpjBKNzuNz6E1ak15vMth0Eq3RaBACUxeJeGJ5fmz77+97/a5OYVenQCLsEclVMrlBK1UrxVaFIeSWO8DxZFKRSiQ0GGRigVQIYgNXoHttdfPTz54zlRQcdqtUwpEKlKBSIlXJDHopT+F0qAG9LXq5WCoH3AXMkPV0iupFPd6x7e2LV9+PxKNNjT2dbLW4p1urFOtkXJlFLlCC12U3cBgQWHVgGjqNHHiwSFjP4WjeXt3aHv3qs1OuhK6ppZUj4gLUJHKrvBtMXaE3aC0+mxWM22IxqLVqpVwmFYu5FHF35vY321u3//bFOzl3TkNv7e7WqLWAVmqlRsFW6mUChcHg8dl1aqNBLdeoFUqtWtrZyem2fLBxb3Nz9suv/vbzTDAoZDWpgooWgU4p1BjFUo2MJ7VbpSZ30AXsQWGxmjVKhbi7mSPnv7q7D8Lg8p1r58//ACTvznqtRqzT8vkGVY9WpxSJlTq16ECD1oOfIyEDsOSuNl6XqDz/YGt7c2Nt4fbFS7877fdCEnazOGDsFkFSjkkuFClkPJ5EplACspisKqmIxznQtyh16dv1zXsb65sLS1Ojn/9+0JzwNHco2DJIyxOppaJuLhgiSDQ8HohNApC9uthsjsTI8fz5n2u7D7dAv5WF6fGblz46bnbHNfQOlRYAKeZwFCDGiDU9XG43SKFdHZymVoFEIFI6/v7PvZ3tNZDM19eW5+5MTFz49Kcxs8fRLeCxJVqlENKD1KUxKuQCUMPtaunkdnRJpUrvjYffbuzdXVvZ3ADZfHl+dnxy/G+vhvR2M9jGHVy5xqIQ6K1KqQBESX6PkM/lguYKQf/U3XsbIC5vboOwDDquLM0tj1++9MuCy2q0O+TdXU1cNl+j6Gpji/nsjg52j1SpNnDl7369t72xc3dnfXNnY2Nz8+BtYGd2ZvrW6F/fzEAai9WiE4iEMhApJXq+TKPkdwn1YkVP7+K/Nra+3tnZ29raXtvcPKjbWFs7+A17YvzGH1+IW3Q6s0UrP9hCBsihl2shIEPDmasPv965/8362tbm2vYuKAINN7e21vdWl3aWxhfnb3z05ohfqzZKZBK5CkhLrhR3CmIvzu7/a3/z3oP11b0HdzfXN8DLwN7e7vbW9t37e+sbO9sbS4t3zv84o+yqI2EwKAQcgaU18T1PfjB2f3d5enb97sO1je27+1tLsxsb+/t7u1vbWzu7+/s7C7e//O+ngoqOxgYqAUuikskUJAaDxBFY/Jc/mVmcmVtcWF5dA2BugYPu37u7u725vr+/ubl66VfDPjYBgcFTSHgsgUgiEfFEEoWKhhMJguPnV5fuzC7ML23u7OwDHe3d3d/b2dzY3VkZ/+iZYDMNgyORsWgcClZbUwuDozFoMoNEbsAT6MqXRlcm52Zm13f29+/vbO/evbe/t72ztjzxfrybdegYml5HQlbDa6qrqqqqq3AoOIpEILHqGpj4xugHy6sLM3OrW1tbuzt79+7f29vZWbvyuquVgPruEQyNiqmthCPRaAQCVv0f3//Od48cPlJdCWd2UEmuXyyAFD+zuLIMxgj6bW9Mf/6cjEymYWpqwNFQMBgGh0Ui4LCaIzWVFVWVTzyOgNfA6Bxu08/+NrsIMvLS1vYOwHN9+eKTPDKJgMHV4yoPHT5WC0ehao4erYEjqC0t9Y31FMIxNB5Fbmkiys9dXl7Z2N4AI9+fW5u6+aaJRsKRKbBKUAKHo44ga1EEDByNo7HqmPX1zHomvb76O5VUeYPs3KUtIPXN7d3d1Rtjv7B0MMhoLAkJQ2GIGASmAo0mkOitnc1EOoPBZNKZjUQUuur7VQwxWXruwcLM/PrWg8216x+EqTQKFgaH12BgRysraulMBIlJJ9KaOzC0hjoygcxEkSgdjMcOH+sQM2PvbI3Nr6w/WJwYzTRRACQobG0NCgEjEolNjfj6VgaqFt/Y0tTUVN/YwCS2tTaQcUzqf7T3tFk/XZtYWJmfvv6fHdg64rEqOh19cCk8Hken0tpYiGokkS6qJ9JbuW0ECpNZVdPAIh893NUsGJycml26PvFXA4uBq3kcxqIhEMjaJ45WYuqZZEFnbSW5oZFTx6hntXfQiUQWGkHBkhqegJsavb9cmZ7/8uLzzFYauuIQjEZGIWpRKCKzqRVPFIuZVFYTp72Zw6lr6GxubGmnIlCUNsSRysZm0ZN37sxd+IOJ1YakwJ6opoA2jS0sOoPFOFZNYTR21hHqWqjMBjKdgcAwiQRUNQ5WSSM+zm2Pfjw1+5dXW9sba/EwJL2ZjoahauEEPKIWA6upqoHDYTDATzAaNLKWwCRiG5nfh7dQjjZoOP2Xbn2aZLDJ1TQYnMVuwGOQaAQMQyaiiVgkEonBYjFkAhxBQB+htxHRTcSjhGo8pgbPaTF8eO234vrOmmp8NbKFTUehYXgCCkmgEGhkAu5AgEC6ZBwShWumkKiAhngYlkyqYXG6Xr/8SlNPwxHAL0pHC7YKhoIjsVgcnkIkkikUwFgCETRHkFroqPq6Snrn0WoMHvsYgts08EWeKcMfhVWg65rrCGg0GQ0HDKMycEDoRAIOB6eQqyppDWRiMwtPr8fXYFGVaFhFU1voYwutqwpehQAqBziS6tvaaBgsvZFCJuIxGByBTEDW4BoZcHojBtdBOMxsQDxOaqrEd0Hv81pZj4F5o9BEGotJY3Xw2ikoQj2ow2EwBHIzugpfRyeQ6jAYOhWJBTKrwNAeq6qXv9EkYX23mgzgpzPJFCYFiWvtICEwWDyRiCeQyVQsAhgUtZ5MbmDACI3UQ7VUeOV3kQzeU92ShiO4GnoLcB9qY1MHE4Nrb62tplCJGByFhEaj0AQ6q47BaGkgUFvox2j1aCSJAW9riHbySYfQMDoLXV1djaPQmxvw9d00OI1EpFDxSDiaxGxgMZl14PLM+qZ6Wgv2GJbCRLLqQm3diEMoFAxdXVN7rAaJamZTaPxWTB2RyKAia1A4Sh2LTiGTGTR6AyC4uBUOIyMqaxpp/g4h/CgKVwGvxRGBQcBo7XRqdyuOQQYuCkNRG+uoFBqdRmYCD21gtIi6aCRSzROVLXgXW42pgKGxRBSRAr5rSY0UKqcJgadQsHAUtaGehgcQgfnjSMzGOhpN2EVDMGkoFsrOMROPVCBQRAyRisMTamiNeAqHVYMgU9FISj2LSKaSQSWJSCUTyXVMclt3fU1zC4GMd7dbqE9UYhAkLJ6MA+ynNVCYTQTAZxIKx2qgkBn19QwqFRy0joClsOpInB4EGo3BNyabzPQn4NS6RhwMBscAN2M2tBCr64BNwQkN9WCIgDNkPBpGa2aRCGQSjaNEf/9xBKqnLJfwD7FYza2oYzhMNZ5cX9/USscQaARCZUMnnUqorkEQ6UQqncigUxg0JpEuqa84hEa1nQbv9MfoOGo7s6qqthZFa2pvomGQFDLQYVs9qYEAnJtAweIoWCqTSiMSKGRhU81hHKHjFXeHqgqHgdMZqEo4Ft/U2FqPRxJoWGwVgs2kthDxVDKNRSHjMYyDbzKD0N1YdQRPELzf26lCANdAkehkSh2zGVAST6DgYJTq2k4Wq45IaWAwGsFt0SwmHizBOlIno6Iah3Vceq1H2lZJYOGpjUxmA5WCQBIIB95AqqhuB/DhySwqmY7HkdAUKgF4PYXcTK8lkIinZ38PiQ34Q0gUCY/AkREY8AWUBydjanFdDUwyHk8HIJKodSgClUGHHarE0UhoBqPj1xufDcuglsefqKiuOFqDrEYTyFQaAY1loMnkttZ6EhlHBv0BmFgynU6uOXwUg8ZimS2+67vn31JIepDogyoYDIkj40lMCgpHQDVgaZ1NKCYODzZ9FQqNYtKQsOpaJBmBwtH5zyzvf/yH4R4buebY99DA044CCQDbAn5GRdcQ2tqoTAaTTgKVtTA0CQ+rQaCQdbVUqv7qzv6FL99x6ERPfLfqSDVw+8N4HJKGhxHqAF2auzo7yFQmcA8iFo8jkrDw6hosqYaAa399dXv/5rXzg1IZ67HvHarB1yDxZCS+kVBLxaDwNDq1vgusMDK1jkFBYvFgT2FQOOT3aHj3rUe7+9NXrr/r5Rno/+cQAskgEuA4OoVEIVUQAGuITGZ7RxOVSKeTYBg8WMBUfHU1DNPz3v6j/b3l8akvf2RTChHfOcwSEB+DUUkoCgMF/B2Ga23GM9o4bY0NDDyVwaSSAR2OwSmUszvf/uPu/ubKzNW/npUJ2McOseowFAoMziTByAemTgF+SqpvPVi5jXUsJhmLqD6GR1DDdx49/Ob+ve21hdGrv+1VcxuIiCPYblYtmEAlA4cGj8DhqASwIQiUuibgpihENQzJYlj/9ugfX//j6wfbCxtLE1++F2mVdB+tJlQRmui1SBIaUQMj1gH7wyAQwLTgYOHUghgEbEf+q4f3Hz3a+Prr7Y2VhdnZq3/IsdlCyv89Sm9pamuAfR9XT6PSmK1UYI6VFRUVh4nVlWgsmsgyffLwwcOHd8H9drbWl5eWJs9/eE7Tyul4AgZHUlhkJAl3tAasJTQOUQVyBhKGxNbA6XUduiv3/7X3zaO9zbv7uztbIGCPT4z++ay6pQvkiYrDcHoDFVlNbQCqq6iA4YB50uBoYmNbR3nm3oPt7Yff3Lu7v7+7u7O9vbO4PnXl/A9dPQ0UKrn68LEjeAyptYVMYCBrKipAHmEcnJn982+/vbd3/9tv9vf2QUre2wWh8MHa+sr4V789beuiEPFY1DECnlrHpNZ1gJiGrTyKpNMaGeGpR/cePnrw9TcP7t/d/P91e/s7G9tbi3dGwYuZqZWCB16NxGIIdApIKFgSq6Gurqf89/17e98+urt39+7u3ftbIETu7e3dvb/9YGfzHztz24ujH76UUjQx67AgK+NrkWDTkLCNrqcmd/55b/ebR/f2v/7ng38fcO//Ab7N8uQ=

radius={(w>>1)+2}

store. coin

foreach {
 $coin
 
 +matchpatch[0] [1],16,16,1,10,0,0,1
 channels[-1] 2
 top_left_coordinates={[xm,ym]}
 remove[-1]
 echo $top_left_coordinates
 
 eval radiuses=vector(#2,$radius);coordinates=[$top_left_coordinates];coordinates+=radiuses>>1;ellipse(#0,coordinates+[3,3],radiuses,0,1,255,0,255);
}

For [0] image:

Next step is to inpaint, then insert that 2 cm thingie.

2 Likes

Features on the coin face get buried in the noise. For the terminally curious, alternate the pipeline with ‘display’ stops (d , between every command) and single step through the pipeline. The score channel in the correspondence map produced by matchpatch (blue: channel 2) mainly have pel values in the millions, except around the coin, where scores drop by an order of magnitude (see comparison in the fill pel processor). But there is too much noise to distinguish coin features. Noise could be a big problem with this approach; that’s why I am interested in @Silvio_Grosso playing around with other images. This particular image may be the lucky one. But if some of his botanicals are black-eyed susans, this approach might detect all their eyes and run off the road with a dozen matches.

I expect this can be done entirely with a G’MIC script.

G’MIC includes programming constructs (loops, conditionals, etc) that ImageMagick (IM) does not include. Doing the job with IM also needs some work in a shell script (eg bash or Windows BAT). However, I know IM better than G’MIC, so that is how I would do it.

As usual, there are many possible methods. Overall, we need to (a) find the coin, (b) infill the coin, (c) make a 2cm scale slightly smaller than the coin and (d) composite that over the image.

(Wikipedia says a 5 euro cent coin is 21.25mm diameter.)

What visually distinguishes the coin? It is a dark circle on a white background. In the OP example, it is also a distinct colour. I won’t rely on that, because an image might have dark brown twigs the same colour as the coin.

So I assume the coin is the only dark roughly circular object. I also assume the coin never overlaps with plants.

I threshold the image so we get black objects on a white background. Then I list each black object, the “connected components”. The bounding box of the coin will be roughly square. I will assume that only one bounding box is roughly square. If there are many, we could also compare the size (in pixels) of the component with the area of the bounding box; if the component is roughly pi * r^2 then we assume it is circular.

The -morphology denoises.

For inpainting, we expand the bounding box by 10 pixels in each dimension, so we include the shadow of the coin.

Windows BAT script:

set Infile=coinPlant.jpeg
set Outfile=coin2cm.jpeg

set NumSq=0

for /F "usebackq tokens=1-5 delims=: " %%A in (`%IMG7%magick ^
  %Infile% ^
  -threshold 50%% ^
  -morphology open disk ^
+write x1.png ^
  -define "connected-components:verbose=true" ^
  -define "connected-components:area-threshold=1000" ^
  -connected-components 8 ^
  info:`) do (
  if "%%E"=="srgb(0,0,0)" (
    for /F "tokens=1-4 delims=x+" %%G in ("%%B") do (
      set WW=%%G
      set HH=%%H
      set XX=%%I
      set YY=%%J
      echo GHIJ= %%G %%H %%I %%J
      echo WHXY= !WW! !HH! !XX! !YY!
      set /A HHlo=!HH!*90/100
      set /A HHhi=!HH!*110/100
      if !WW! gtr !HHlo! if !WW! lss !HHhi! (
        echo Squarish
        set /A NumSq+=1
        set /A coinXX=!XX!
        set /A coinYY=!YY!
        set /A sqWW=!WW!+20
        set /A sqHH=!HH!+20
        set /A sqXX=!XX!-10
        set /A sqYY=!YY!-10
        set /A twoCm=!WW!*2000/2125
      )
    )
    echo %%A %%B %%C %%D %%E %%F
  )
)

echo NumSq=%NumSq%

if not %NumSq%==1 (
  echo %0: %Infile% found %NumSq% squarish components.
  exit /B 1
)

echo Found one squarish bounding box: %sqWW% x %sqHH% + %sqXX% + %sqYY%
echo twoCm = %twoCm%

rem Make the coin transparent.

%IMG7%magick ^
  %Infile% ^
  ( +clone ^
    -fill White -colorize 100 ^
    -fill Black -draw "translate +%sqXX%+%sqYY% rectangle 0,0,%sqWW%,%sqHH%" ^
+write x2.png ^
  ) ^
  -alpha off ^
  -compose CopyOpacity -composite ^
  coinTrans.png

rem Fill the hole

%IM7DEV%magick ^
  coinTrans.png ^
  -process 'fillholes wr 3 lsr 50 cp onepixel auto_repeat verbose' ^
  -process 'fillholes wr 3 lsr 50 cp onepixel auto_repeat verbose' ^
+write x3.png ^
  ( -size %twoCm%x30 xc:None ^
    -fill Black ^
    -draw "rectangle 0,2,%%[fx:w-1],4" ^
    -draw "rectangle 0,0,0,6" ^
    -draw "rectangle %%[fx:w-1],0,%%[fx:w-1],6" ^
    -draw "rectangle %%[fx:w/2],0,%%[fx:w/2],6" ^
    -pointsize 20 ^
    -gravity Center -annotate +0+0 "2 cm" ^
+write x4.png ^
  ) ^
  +gravity ^
  -geometry +%coinXX%+%coinYY% ^
  -compose Over -composite ^
  %Outfile%

Each line like “+write xN.png” is only for debugging, and can be removed.

This uses my Process module fillholes, which needs IM to be rebuilt. If you don’t want to rebuild IM, my page Filling holes shows alternative methods.

The result, coin2cm.png, is:

1 Like

I see. Could it detect if pixels around the “supposed coin” are the same color as the background? ( Didn’t want to say white here, it could be a very light grey, or any color, or noisy). I guess it would still lead to false positives…

Hello @grosgood

Just tried on Windows 10 (powershell) - gmic-3.2.4-cli-win64
With a different jpg image (apple1.jpg)

With GIMP I have cropped the coin from this image (apple1.jpg) and exported and renamed as coins.png

Unfortunately, It throws up an error:
PS C:\Users\gross\Downloads\gmic_3.2.4_cli_win64\gmic-3.2.4-cli-win64> .\gmic.exe apple1.jpg coins.png +matchpatch[0] [1],16,16,1,10,0,0,1 channels. 2 fill. ‘i(x,y)<200000?1:0’ +barycenter. e ‘{@0,1}’
[gmic]-0./ Start G’MIC interpreter.
[gmic]-0./ Input file ‘apple1.jpg’ at position 0 (1 image 800x533x1x3).
[gmic]-1./ Input file ‘coins.png’ at position 1 (1 image 141x126x1x4).
[gmic]-2./ Estimate correspondence map between image [0] and patch image [1], using 16x16x1 patches, 10 iterations, 0 randomizations and occurrence penalization 0 (score returned).
[gmic]-2./ *** Error *** Command ‘matchpatch’: [instance(800,533,1,3,000001b0d6987040,non-shared)] gmic::matchpatch(): Instance image and specified patch image (141,126,1,4,000001b0d6706a70) have different spectrums.

To sum up.
From my understanding of this command:

  • At present, with GIMP, I should add manually the graduated scale (2 cm) on every image present in a folder.
  • Afterwards, with your G’mic script, I can automatically remove the coins from all the images.
    Is it correct?

Hello @snibgo

Just installed ImageMagick on my computer with Windows 10 (ImageMagick-7.1.1-8-Q16-HDRI-x64-dll.exe).

Thanks a lot.
I am going to try your .bat script
If I understand it correctly it should even automatically add the graduated scale (2 cm)

Hello everyone,

In case you are interested here are 4 .jpg images to test (zip folder - Dropbox):
images

There is one (coins.jpg) which might be tricky because there are plenty of coins :slight_smile:

Remove the alpha channel from coins.png. That’s the ‘different spectrums’ error. G’MIC, in its charming way, calls the aggregate number of channels the image’s spectrum.

Here’s the rough workflow

  1. devise a mask of your ‘benchmark euro’ Use that for the entire batch. It could look something like my 64,64,1,3 mask of your euro. I just cropped it from your botanical.
  2. Iterate over the batch, using something like my pipeline . For each image:
  3. The third image in the list has been derived from the correspondence map to isolate the coin. Let us call it the ‘correspondence mask’
  4. Use one copy of the correspondence mask as a mask for inpaint_morpho. to blot out the coin in your botanic image - select the botanic, pass the mask as an argument.
  5. -input your centimeter scale from wherever you stash your centimeter scales. That, having an alpha channel, can be alpha-composed onto the botanical image. The barycenter coordinates identify the center of the coin, Or look at @Reptorian 's script: he locates the upper left corner of the image (he doesn’t use barycenter)
  6. Use image to composite the scale and the botanical image. The centimeter scale would be the ‘sprite’ image and the botanical the selected image. The x,y coordinate arguments to image locates the upper-lefthand corner of the sprite. depth, z, and channel, c, are zero and opacity is = 1.
  7. Output the composite botanical+centimeter scale and move onto the next botanical, reusing the coin mask. -remove any other intermediary image (correspondence mask and such)

I think 1-7 can be wholly automatic; but NB my black-eye susan comment made to @prawnsushi. Some images may break. When I get home and after the puppy walk, I think I can script this out.

That’s it for now. Hope this helps.

@grosgood

Remove the alpha channel from coins.png

Thanks. Somewhat, It did the trick :slight_smile:

I have an approach that may work for all cases in context of selection.

apple1.jpg apple2.jpg blueberry.jpg foreach { rgb2lch split c keep[1] gt 20 }

Here’s the problem. Filling those little holes that shows up. After that, it’s just a matter of detecting circle with matchpatch or by area-size selection, and using inpaint, and then using image command to do the 2cm thing.

Right now, I don’t feel like finishing this up, so I will give pointers instead.

Tee, Hee. Did a rather better job of picking the apple rather than the euro, no? I expected that contrary cases could arise; am hoping that contrary cases could be small in number or constrained (ever the optimist). Leaving the office now. Thanks for something to ponder on whilst riding on the subway and walking The Little Mutt…

And thank you for the interesting problem!

Here’s the code:

rep_tool_for_silvio_grosso_detect_and_inpaint_and_insert_2_cm:

foreach {
 +rgb2lch split[-1] c keep[0,-2] gt[-1] 15
 +area[-1] 0,0 
 lt[-1] 900
 if iv
  inpaint_pde[-2] [-1],0,0,0
 fi
 rm[-1]
 area_fg[-1] 0,1
 +gt[-1] 1000 *[-2] [-1] rm.
 +colormap[-1] 0
 for w#-1>4 {
  +gt. {ia} *[-2] [-1] rm. colormap. 0 
 }
 eq[-2] {i(#-1,1)} rm.
 dilate[-1] 6
 inpaint_pde[-2] [-1],100%,2,1
}

Not really a good result, but oh well.

@Silvio_Grosso , what you need here is clearly a circle detector (usually implemented with the Circle Hough Transform (CHT)), as the coins in your images have different sizes and colors.

CHT is not already implemented in G’MIC, but 100% sure it can be done (the classical hough transform for detecting lines is implemented and works as expected).
It could be a nice addition TBH.

Once the coin is detected, using one of the inpaint method of G’MIC should be OK to remove it.