How to replicate inpaint results from within GIMP in GMIC interpreter?


#1

Hi,

I’m experimenting with the G’MIC inpainting algorithms in order to remove artifacts from images I’m working with. I wish to use the G’MIC tool from scripts to automate this work, but struggle to replicate the results of the G’MIC inpainting algorithms as applied from within GIMP when using the G’MIC interpreter. The relevant images for this post can be found here as there is a restriction on new users of 4 images per post: http://imgur.com/a/duajD

The first image shows the original image, and the second image shows the mask I’ve added in red.

From within GIMP 2.8.16 with G’MIC 1.7.9, I select the mask using the fuzzy select tool without anti-aliasing. I then apply the “Inpaint [multi-scale]” G’MIC filter to the image with the mask selected, using the default filter settings. This gives the following result (repeated 5 times, resetting back to the mask before re-applying the filter): (see imgur)

It is apparent in these images when the mask is selected in GIMP, more local details are used by the G’MIC inpainting filter. Following this, I then apply the “Inpaint [multi-scale]” G’MIC filter to the image without the mask selected, using the default filter settings. This gives the following result (repeated 5 times, resetting back to the mask before re-applying the filter): (see imgur)

From these images it seems that the inpainting algorithm uses more global information when the mask is not selected. At last, I try to apply the same inpainting algorithm from the command line:

gmic wallet_red_mask.png --select_color 0,255,0,0 -inpaint_patchmatch[0] [1] -output[0] wallet_gmic.png

This command is ran five times, resulting in the following five images: (see imgur)

It appears that the inpainting algorithm uses global information in this case as well, but that it might be more unstable than when used from within GIMP.

How can I replicate the GIMP results for when the mask is selected, such that more local information is used, when using the G’MIC interpreter from the command line? I’ve tried using crops of the input image without being able to replicate the GIMP results well.


#2

I experimented more with using cropped versions of the masked image as input. Instead of feeding the full image with the mask as input, I ended up using a cropped version of the image with a width and height both equal to twice the width and height of the mask, centered on the mask. The reason why I ended up with poor results earlier was that my crops were too large. Through experimentation, the results from the new crop dimensions matches the output from GIMP closely. Any further hints for improvements are appreciated.


(Mica) #3

The gmic GUI in GIMP can be set to show you the command it runs, set the outputs to verbose.


#4

I’ve tried studying the output using the “very verbose (console)” option, and in the two cases the output is identical (besides a different number of scales).

Output for inpainting with the mask selected:

[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./gimp_filter_sources/*if/ Set local variable local='/home/pveierland/.config/gmic/update179.gmic'.
[gmic]-0./gimp_filter_sources/*if/*if/ Import custom commands from file '/home/pveierland/.config/gmic/update179.gmic' (added 2919 commands, total 4809).
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/ Input image at position 0, with values (47,104,111,109,101,47,112,118,101,105(...),108,97,110,100,47,46,103,109,105,99) (1 image 22x1x1x1).
[gmic]-1./gimp_filter_sources/*if/*if/gimp_filter_sources/ Input image at position 1, with values (104,116,116,112,58,47,47,103,109,105,(...)116,101,49,55,57,46,103,109,105,99,1) (1 image 30x1x1x1).
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/ Set local variable file_hotchocolate='/home/pveierland/.config/gmic/hotchocolate.gmic'.
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/ Input raw file '/home/pveierland/.config/gmic/hotchocolate.gmic' with type 'uchar' at position 0 (1 image 1x1x1x1).
[gmic]-1./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/ Set local variable c='32'.
[gmic]-1./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/ Remove image [0] (0 images left).
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*if/ Set local variable is_too_old='0'.
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/ Remove image [] (0 images left).
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/ Input image at position 0, with values (47,104,111,109,101,47,112,118,101,105(...)111,108,97,116,101,46,103,109,105,99) (1 image 47x1x1x1).
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Set local variable strin1='#@gimp : Sep = separator(), note = not(...)erle.users.greyc.fr">David Tschumperl'.
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Set local variable strin2='#@gimp : Sep = separator(), note = not(...)erle.users.greyc.fr">David Tschumperl'.
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Set local variable strout='#@gimp : Sep = separator(), note = not(...)p://goo.gl/PTcydp</a> ]</i></small>"}'.
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Set local variable filename='/home/pveierland/.config/gmic/update179.gmic'.
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Input raw file '/home/pveierland/.config/gmic/update179.gmic' with type 'uchar' at position 0 (1 image 32x1x1x1).
[gmic]-1./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Split image [0] in 'discard' mode, according to value sequence '10'.
[gmic]-3./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Set local variable is_marked='0'.
[gmic]-3./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Remove images [0,1,2] (0 images left).
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Remove image [] (0 images left).
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/ *** Error (file '/home/pveierland/.config/gmic/update179.gmic', line #15251) *** Command '-i': gmic::fopen(): Failed to open file '/home/pveierland/.config/gmic/gimp_filter_sources' with mode 'rb'.
[gmic]-3./ End G'MIC interpreter.

[gmic_gimp]./update/ Filter file '/home/pveierland/.gmic' not found.

[gmic_gimp]./preview/ -v 0 -gimp_inpaint_patchmatch_preview 0,9,10,5,1,255,0,0,255,0,0
[gmic]-1./ Start G'MIC interpreter.
[gmic]-1./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/ Set local variable R='255'.
[gmic]-1./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/ Set local variable G='0'.
[gmic]-1./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/ Set local variable B='0'.
[gmic]-1./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/ Set local variable A='255'.
[gmic]-1./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/ Select color (255,0,0,255) in image [0], with tolerance 0.
[gmic]-2./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/*if/ Dilate image [1] with kernel of size 3 and neumann boundary conditions.
[gmic]-2./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/ Set random generator seed to 0.
[gmic]-1./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/ Inpaint image [0] with mask [1], using a multiscale patchmatch algorithm with auto-scales, 9x9 patches, 10 iterations per scale and blending size 5.
> Process scale 1/4 -> 12.5%
> Process scale 2/4 -> 25%
> Process scale 3/4 -> 50%
> Process scale 4/4 -> 100%
[gmic]-2./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/ Remove image [1] (1 image left).
[gmic]-1./ End G'MIC interpreter.

[gmic_gimp]./apply/ -v 0 -gimp_inpaint_patchmatch 0,9,10,5,1,255,0,0,255,0,0
[gmic]-1./ Start G'MIC interpreter.
[gmic]-1./gimp_inpaint_patchmatch/*repeat/*local/ Set local variable R='255'.
[gmic]-1./gimp_inpaint_patchmatch/*repeat/*local/ Set local variable G='0'.
[gmic]-1./gimp_inpaint_patchmatch/*repeat/*local/ Set local variable B='0'.
[gmic]-1./gimp_inpaint_patchmatch/*repeat/*local/ Set local variable A='255'.
[gmic]-1./gimp_inpaint_patchmatch/*repeat/*local/ Select color (255,0,0,255) in image [0], with tolerance 0.
[gmic]-2./gimp_inpaint_patchmatch/*repeat/*local/ Set random generator seed to 0.
[gmic]-1./gimp_inpaint_patchmatch/*repeat/*local/ Inpaint image [0] with mask [1], using a multiscale patchmatch algorithm with auto-scales, 9x9 patches, 10 iterations per scale and blending size 5.
> Process scale 1/4 -> 12.5%
> Process scale 2/4 -> 25%
> Process scale 3/4 -> 50%
> Process scale 4/4 -> 100%
[gmic]-2./gimp_inpaint_patchmatch/*repeat/*local/ Remove image [1] (1 image left).
[gmic]-1./ End G'MIC interpreter.

Output for inpainting without the mask selected:

[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./gimp_filter_sources/*if/ Set local variable local='/home/pveierland/.config/gmic/update179.gmic'.
[gmic]-0./gimp_filter_sources/*if/*if/ Import custom commands from file '/home/pveierland/.config/gmic/update179.gmic' (added 2919 commands, total 4809).
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/ Input image at position 0, with values (47,104,111,109,101,47,112,118,101,105(...),108,97,110,100,47,46,103,109,105,99) (1 image 22x1x1x1).
[gmic]-1./gimp_filter_sources/*if/*if/gimp_filter_sources/ Input image at position 1, with values (104,116,116,112,58,47,47,103,109,105,(...)116,101,49,55,57,46,103,109,105,99,1) (1 image 30x1x1x1).
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/ Set local variable file_hotchocolate='/home/pveierland/.config/gmic/hotchocolate.gmic'.
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/ Input raw file '/home/pveierland/.config/gmic/hotchocolate.gmic' with type 'uchar' at position 0 (1 image 1x1x1x1).
[gmic]-1./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/ Set local variable c='32'.
[gmic]-1./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/ Remove image [0] (0 images left).
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*if/ Set local variable is_too_old='0'.
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/ Remove image [] (0 images left).
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/ Input image at position 0, with values (47,104,111,109,101,47,112,118,101,105(...)111,108,97,116,101,46,103,109,105,99) (1 image 47x1x1x1).
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Set local variable strin1='#@gimp : Sep = separator(), note = not(...)erle.users.greyc.fr">David Tschumperl'.
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Set local variable strin2='#@gimp : Sep = separator(), note = not(...)erle.users.greyc.fr">David Tschumperl'.
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Set local variable strout='#@gimp : Sep = separator(), note = not(...)p://goo.gl/PTcydp</a> ]</i></small>"}'.
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Set local variable filename='/home/pveierland/.config/gmic/update179.gmic'.
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Input raw file '/home/pveierland/.config/gmic/update179.gmic' with type 'uchar' at position 0 (1 image 32x1x1x1).
[gmic]-1./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Split image [0] in 'discard' mode, according to value sequence '10'.
[gmic]-3./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Set local variable is_marked='0'.
[gmic]-3./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Remove images [0,1,2] (0 images left).
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/_gimp_filter_sources/*if/*local/ Remove image [] (0 images left).
[gmic]-0./gimp_filter_sources/*if/*if/gimp_filter_sources/*local/ *** Error (file '/home/pveierland/.config/gmic/update179.gmic', line #15251) *** Command '-i': gmic::fopen(): Failed to open file '/home/pveierland/.config/gmic/gimp_filter_sources' with mode 'rb'.
[gmic]-3./ End G'MIC interpreter.

[gmic_gimp]./update/ Filter file '/home/pveierland/.gmic' not found.

[gmic_gimp]./preview/ -v 0 -gimp_inpaint_patchmatch_preview 0,9,10,5,1,255,0,0,255,0,0
[gmic]-1./ Start G'MIC interpreter.
[gmic]-1./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/ Set local variable R='255'.
[gmic]-1./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/ Set local variable G='0'.
[gmic]-1./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/ Set local variable B='0'.
[gmic]-1./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/ Set local variable A='255'.
[gmic]-1./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/ Select color (255,0,0,255) in image [0], with tolerance 0.
[gmic]-2./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/*if/ Dilate image [1] with kernel of size 3 and neumann boundary conditions.
[gmic]-2./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/ Set random generator seed to 0.
[gmic]-1./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/ Inpaint image [0] with mask [1], using a multiscale patchmatch algorithm with auto-scales, 9x9 patches, 10 iterations per scale and blending size 5.
> Process scale 1/5 -> 6.25%
> Process scale 2/5 -> 12.5%
> Process scale 3/5 -> 25%
> Process scale 4/5 -> 50%
> Process scale 5/5 -> 100%
[gmic]-2./gimp_inpaint_patchmatch_preview/gimp_inpaint_patchmatch/*repeat/*local/ Remove image [1] (1 image left).
[gmic]-1./ End G'MIC interpreter.

[gmic_gimp]./apply/ -v 0 -gimp_inpaint_patchmatch 0,9,10,5,1,255,0,0,255,0,0
[gmic]-1./ Start G'MIC interpreter.
[gmic]-1./gimp_inpaint_patchmatch/*repeat/*local/ Set local variable R='255'.
[gmic]-1./gimp_inpaint_patchmatch/*repeat/*local/ Set local variable G='0'.
[gmic]-1./gimp_inpaint_patchmatch/*repeat/*local/ Set local variable B='0'.
[gmic]-1./gimp_inpaint_patchmatch/*repeat/*local/ Set local variable A='255'.
[gmic]-1./gimp_inpaint_patchmatch/*repeat/*local/ Select color (255,0,0,255) in image [0], with tolerance 0.
[gmic]-2./gimp_inpaint_patchmatch/*repeat/*local/ Set random generator seed to 0.
[gmic]-1./gimp_inpaint_patchmatch/*repeat/*local/ Inpaint image [0] with mask [1], using a multiscale patchmatch algorithm with auto-scales, 9x9 patches, 10 iterations per scale and blending size 5.
> Process scale 1/7 -> 1.5625%
> Process scale 2/7 -> 3.125%
> Process scale 3/7 -> 6.25%
> Process scale 4/7 -> 12.5%
> Process scale 5/7 -> 25%
> Process scale 6/7 -> 50%
> Process scale 7/7 -> 100%
[gmic]-2./gimp_inpaint_patchmatch/*repeat/*local/ Remove image [1] (1 image left).
[gmic]-1./ End G'MIC interpreter.

Using debugging output will likely explain the difference, but the amount of debug output is massive and difficult to read.


#5

I believe @paperdigits was referring to the G’MIC filter in GMIP,

from which you can determine the particular command’s arguments by the new layer’s name. I haven’t used -inpaint for practical use yet but I suggest you control how much and which surrounding areas the command should use. Try using -inpaint because that is the native (console) command. The other inpaint commands are derived from this one.


Colored pencil filter internals
#6

Using the verbose (layer name) option for the G’MIC filter within GIMP results in the following layer name:

[G'MIC] Inpaint [multi-scale]: -gimp_inpaint_patchmatch 0,9,10,5,1,255,0,0,255,0,0

Which is the same information which is available through the verbose log output. However this layer name is identical for the case where the mask is selected, as for the case when the mask is not selected. Therefore it does not reveal any more information which explains the different results for when the mask is selected versus when it is not selected.


#7

I am using the latest versions of GIMP and G’MIC. The default arguments appear to be the same but I could not tell you if there have been changes under the hood. Try updating and see if it helps. Hope you find a solution :slight_smile:


(David Tschumperlé) #8

Maybe some additional information about this.
When you do a selection in GIMP, what is given to the G’MIC plug-in is only the image corresponding to the bounding box of that selection. So the inpainting filter is able to find image information only in that bounding box (so if your selection is perfectly rectangular, it won’t work at all).

Using selections with the inpainting filter is a smart way to restrict the search area for the image reconstruction, because sometimes other (wrong) regions of images can match the similarity function and may be copied in the region to reconstruct.

If you use the command line, then yes, the best way to do the same is to perform the filter on a pre-cropped image, just as you did.


#9

Thank you for the precise answer, and for the inpainting algorithm and implementation! :smiley: