So I figured out how to do it, I was almost there except that ‘_max_opacity_mask’ is undocumented on the reference of the ‘image’ command.
So here’s the key information:
definition of the ‘image’ command:
image (+): [sprite],_x,_y,_z,_c,opacity,[opacity_mask], _max_opacity_mask
And the info missing:
What is _max_opacity_mask ?
That’s the value defined in [sprite_mask] for which the opacity of the drawing will be 100%. Usually, its value is chosen to be 1 (default) or 255 (when dealing with alpha channels from RGBA images for instance).
“When the opacity argument is just a scalar, I think it’s better to have a fixed range for the opacity ([0,1] is good enough). In the case of an opacity mask, it would be not convenient nor efficient to be forced to renormalize it before being used, that’s why there is this extra argument for command image.”
So the conclusion is to put the mask in ‘255’ max_opacity_mask mode, so image[background] [sprite],0,0,0,0,1,[mask],255
So the grayscale image which has values of between 0 and 255 can be applied to opacity which is normally 0 to 1 in fractions.
Or to put the gray depthmap in C=3 (alpha channel), first do a to_rgba on the background and put the depthmap and mask in the appropriate modes then do image[background] [sprite],0,0,0,3,1,[mask],255 I guess. Or skip the mask part altogether there I suppose.
info was found on Use G'MIC to combine separate R,G,B images to create RGB image
I also read there that colorspace conversion in G’MIC is done with fractional values of 0 to 255 rather than a much larger range of integers. So keep that in mind too with G’MIC.