Scripted foreground extraction


I’ve tried this tutorial and I think it gives great results:

Compared to grabcut, I think it’s easier to set up a few points than to draw lines and curves. I would like to integrate this with another software. Provided I know foreground and background points coordinates, is there a way to script the extraction (using gmic cli, not gimp) ?

Thanks for your kind help

1 Like

Yes, that’s possible. Basically G’MIC uses a simple watershed technique with a specific potential map, nothing really fancy.
So the main difficulty here is to provide an image where non-zero pixels are your labeled points (e.g. 1=background and 2=foreground). Once you have this, do something as:

$ gmic input.jpg labeled_points.png +_x_segment[0] +watershed[1] [-1]

and you’ll get the full labeled result as the last image.
The command _x_segment is the one used by the G’MIC filter to estimate the potential map from the input image.

Neat ! Looks like it won’t be too much work to integrate. I’ll try this and get back with my results.

Very clear explanations, thank you.


I’m getting great results with fewer interactions and very reasonable time given the image size.

I could probably squeeze a bit more time by calling the C lib ?

Yes, I would say the C++ lib is maybe easier to use :
Just feed gmic() with a list of two input images, and use the same command as the command string.

Our current stack is python/numpy-based, so calling C++ libs is more involving than calling C libs. It should be possible though, I’ll give it a try.

There is also a C API for the G’MIC library, which works basically the same.
Look at these sources for more info and examples of use : gmic-community/libcgmic at master · dtschump/gmic-community · GitHub

Ho-hum: maybe I should grit my teeth and learn C(++) some day so that my scripts aren’t so darn slow.

Sorry I didn’t update this thread. Here’s what I did to try to interface gmic with python:

  • compile gmic_libc

  • generate bindings to gmic_libc.h with ctypesgen (GitHub - davidjamesca/ctypesgen: Pure-python wrapper generator for ctypes). This automatically converts the structs and the functions to a python module, linked to the .so file.

  • test calling gmic from python. Example, unrelated to my extraction problem:

    import gmic
    from ctypes import POINTER, c_bool, c_float, c_uint


    p_bool = POINTER(c_bool)
    p_float = POINTER(c_float)

    options = gmic.gmic_interface_options()
    options.ignore_stdlib = False
    options.p_is_abort = p_bool(c_bool(False))
    options.p_progress = p_float(c_float(0.0))
    options.interleave_output = False
    options.no_inplace_processing = True
    options.output_format = E_FORMAT_FLOAT

    image = gmic.gmic_interface_image() = “test”.encode(“utf-8”)
    image.width = 500
    image.height = 500
    image.spectrum = 4
    image.depth = 1
    image.is_interleaved = False
    image.format = E_FORMAT_FLOAT

    command = ‘v 0 apply_channels “div 2”,rgba_r polaroid 5,30 rotate 20 drop_shadow , drgba display’

    gmic.gmic_call(command, c_uint(0), image, options)

So far, so good, it works fine. BUT… I decided not to go this way because in my case the small performance improvement comes at the cost of more boilerplate and above all a packaging problem (it’s just easier to rely on the already available CLI version of GMIC). For my use case it’s just not worth it, but should I need python bindings it would be a good solution to use ctypesgen and maybe a light bridge between gmic_interface_image and numpy.

Thanks again for your help.

This was a great reference! @David_Tschumperle out of curiosity, where is the source code of fx_extract_foreground ?

I spent a while figuring out how to create the image with labeled points. So here is my code for reference. I’m using the Python bindings:

import gmic
import numpy
from enum import Enum

class Labels(Enum):
    ZERO = 0

g = gmic.Gmic()
image_results = []"input input.jpg", image_results)
input_image = image_results[0]

zero_image = gmic.GmicImage(width=input_image._width,
                            height=input_image._height, spectrum=1)

# Have to copy the array because the to_numpy() result is read-only
arr = zero_image.to_numpy().copy()

bg_points = [(967, 1651), (1656, 1588), (2068, 3636), (649, 3675),
             (1107, 2093)]

fg_points = [(1323, 1665), (1173, 1710), (1167, 1879), (1110, 2263),
             (933, 2331), (630, 2671), (757, 2506), (1230, 3600),
             (964, 3487), (1236, 1788), (1065, 2281), (1217, 2173),
             (1140, 1842), (991, 2231)]

for bg_x, bg_y in bg_points:
    arr[bg_x, bg_y] = Labels.BACKGROUND.value

for fg_x, fg_y in fg_points:
    arr[fg_x, fg_y] = Labels.FOREGROUND.value

labeled = gmic.GmicImage.from_numpy(arr)

# Do permutation here because using GmicImage.from_numpy_helper(arr
# permute='yxzc') is not working."permute[1] yxzc", image_results)"+_x_segment[0] +watershed[1] [-1]", image_results)"display", image_results)

Here is the result:

It’s found on github/gmic/src/gmic_stdlib.gmic.

Here’s the extracted code:

#@gui Extract Foreground [Interactive] : fx_extract_foreground, gui_no_preview
#@gui : Feathering = _float(0,0,4)
#@gui : Dilation = int(0,-32,32)
#@gui : Output Mode = choice{3,"RGBA Image (Full-Transparency / 1 Layer)","RGBA Image (Updatable / 1 Layer)",
#@gui : "RGB Image + Binary Mask (2 Layers)","RGBA Foreground + Background (2 Layers)"}
#@gui : View Resolution = _choice{1,"Small (Faster)","Medium","High (Slower)","Very High (Even Slower)"}
#@gui : sep = separator()
#@gui : note = note{"<small><b>Description:</b>\n
#@gui : This filter allows to quickly extract foreground objects from background in opaque RGB images.
#@gui : Click on the <i>Apply</i> or <i>OK</i> buttons below to open the interactive window and start adding
#@gui : foreground and background control points. When you're done, exit the interactive window: your extracted
#@gui : foreground will be transferred back to the host software.\n\n
#@gui : If you are not satisfied with the result, click on <i>Apply</i> once again to modify your control points
#@gui : defined previously. To remove all control points, click on the <i>Reset</i> button above.
#@gui : </small>"}
#@gui : Last Image Size = value(0,0)
#@gui : Control Points = value(-1)
#@gui : sep = separator()
#@gui : note = note{"<small><b>Interactions:</b>\n
#@gui : Use the following actions in the interactive window to build your extraction mask :\n\n
#@gui : - <b>Left mouse button</b> or key <b>F</b> create a new <b>foreground</b> control point
#@gui : (or move an existing one).\n
#@gui : - <b>Right mouse button</b> or key <b>B</b> create a new <b>background</b> control point
#@gui : (or move an existing one).\n
#@gui : - <b>Mouse wheel</b>, or keys <b>CTRL+arrows UP/DOWN</b> zoom view in/out.\n
#@gui : - Key <b>SPACE</b> updates the extraction mask.\n
#@gui : - Key <b>TAB</b> toggles background view modes.\n
#@gui : - Key <b>M</b> toggles marker view modes.\n
#@gui : - Key <b>BACKSPACE</b> deletes the last control point added.\n
#@gui : - Key <b>PAGE UP</b> increases background opacity.\n
#@gui : - Key <b>PAGE DOWN</b> decreases background opacity.\n
#@gui : - Keys <b>CTRL+D</b> increase window size.\n
#@gui : - Keys <b>CTRL+C</b> decrease window size.\n
#@gui : - Keys <b>CTRL+R</b> reset window size.\n
#@gui : - Keys <b>ESC</b>, <b>Q</b> or <b>ENTER</b> exit the interactive window.
#@gui : </small>"}
#@gui : sep = separator()
#@gui : note = note("<small>Author: <i>David Tschumperlé</i>.      Latest Update: <i>2014/29/09</i>.</small>")
fx_extract_foreground :
  if !$! return fi
  foreach {
    => "[G"{`39`}"MIC] Interactive Foreground Extraction"
    if [$6][0]==-1" || "[$5]!=[w,h] _gui_control_points= else _gui_control_points=$6 fi
    status=${x_segment\ $resolution}
    sh 3 b. $1% if $2>0 dilate. {1+2*$2} elif $2<0 erode. {1-2*$2} fi
    if $3==1 sh 3 max. 1 rm.
    elif $3==2 s c,-3 r. 100%,100%,1,4 rv =>[0] "name(Mask)" =>[1] "name("$nm")"
    elif $3==3
      sh.. 0,2 +channels... 3,3 >=. 3 *[-2,-1] rm.
      sh. 0,2 +channels.. 3,3 <=. {255-3} *[-2,-1] rm.
      sh. 3 *. -1 +. 255 rm.
      pos0=${gui_layer_pos[0]} pos1=${gui_layer_pos[1]}
      =>[0] "name("$nm" [foreground]),pos("$pos0")"
      =>[1] "name("$nm" [background]),pos("$pos1")"
  if narg($status)>=4 u \{$1\}\{$2\}\{$3\}\{$4\}\{{w},{h}\}\{$status\} else u "" fi
1 Like

Thank you very much. This is useful! I was not finding it online because the file is too big for github to index it for search or direct display. My fault for not cloning the repo and check locally :slight_smile:

I bookmark this for easy access: