AI object mask invert — solved by rasterizing (version gap + rasterize-click quirk)

Title: AI object mask invert — solved by rasterizing (version gap + rasterize-click quirk)

Version: darktable-5.6.1-win64

First, a genuine thank you to the developers. AI masking is remarkable functionality to have in a free, open-source raw processor, and I’m more than happy with what’s already been delivered — this is more than I ever expected to have available. The notes below are offered in that spirit, just to help others and to flag one possible quirk.

What I was trying to do:
Invert an AI object mask (module panel invert button).

What happened on my old install:
The invert button had no effect on AI masks. Regular vector masks inverted fine. The AI mask also never appeared in the mask manager. I assumed this was a bug or a bad setting and reinstalled.

What changed:
After reinstalling to the current version, the masking panel now offers a rasterize option at the top of the centre panel. Using it converts the AI mask into a raster mask, which:

now appears in the mask manager

inverts correctly

can be composed with other raster/parametric/gradient masks

So the original problem wasn’t user error or configuration — the rasterize feature simply wasn’t in my older build.

One quirk worth noting (possible bug):
While exploring, I clicked the raster symbol in the masking section and the AI mask vanished rather than converting. I understand from the GitHub issue about AI mask scratchpad cleanup that mask state may not be properly cleared on mask context change / module reset. Might be related.

Summary:

Invert on AI masks works once the mask is rasterized (current version).

Old versions lack the rasterize option entirely, so invert appears broken.

Clicking the raster symbol (vs. the proper rasterize control) cleared my mask.

Question for devs: Is rasterizing intended to be the required step before inverting an AI mask, or should the invert button work directly on the AI mask in the current build?

Thanks and Best regards

Harshadeva Mutucumarana (An avid user of Darktable!)

I am on DT 5.6 windows 11 and I can create an AI mask in one model and then in another module select the same AI mask to use and invert by clicking on the invert drawn mask option shown in red in my screenshot. BTW, I agree the AI masking is a remarkable function to have.

image

1 Like

I’m on 5.7, but have never had a problem like this.

My experience is that, once right-clicked to convert to a path, it behaves like any other drawn-path mask.

I regularly use inversion, eg for subject/background separation.