I’ve been using the darktable nightly builds now for awhile on my Mac Mini M2 Pro and have been impressed with the AI masking feature. I do mostly landscape photography as a hobby and I’ve found that for certain photos the darktable AI masking has done a better job than some of the paid apps for difficult selections such as for a sky around tree branches.
However, in the past week or so I’ve noticed two problems. First, with the default masking model mask sam2.1 hiera small enabled, the AI masking routine starts analyzing the image and then the message changes to object mask preparation failed. If I switch to mask segnext vitb-sax2 hq, the analysis doesn’t fail. Note that up until recently the default mask worked for me without fail. I updated them a few days ago and afterwards noticed this problem. I did another update yesterday and again today but the default mask model is still failing.
As for the segnext model, I’ve noticed another problem I didn’t run into earlier when using sam2.1 Originally, if I clicked on an object and only part of it was masked, holding down the option key on my mac (I imagine alt on Windows) and thyen clicking on the unselected portion I wanted selected allowed me to add to the existing mask. For example, I was using an image taken at a marina with a forest in the background. If I clicked on the forest after the AI analysis, part of it was selected and up until the recent updates, holding down the option key and clicking an unselected portion of the forest resulted in the selection growing. Sometimes it would take several clicks to get everything, but it worked. With the segnext model enabled, whenever I click a non-selected area while holding down the option key, the previously selected area shrinks! Each time I click, the selection shrinks more. Sometimes, the shrinking causes just a slight reduction in the borders; other times, it changes from a big selection to a really tiny one. I’ve tried every modifier key and even no modifier key, but nothing changes. The selection always shrinks.
Yes, I just did so. Attached below is a txt file containing all the messages in my Mac terminal leading up to the failed analysis with sam2.1.
Also attached is a txt file containing the readout for an image using the segnext model where it did not fail but when I try to add to the mask, the selection keeps shrinking. Note that there doesn’t seem to be any detection of a modifier key. My first tests with darktable AI when sam2.1 worked for me allowed adding to an AI mask selection if I held down the option key while clicking outside an initial selection. Then once the mask was the way I wanted it, I right-clicked on the selection to turn it into a mask.
I thought of trying a work-around by doing multiple AI masks, but they seem to overlap rather then add to each other.
Thanks Anddrii! The ai_v*CoreML*` gave me a “no matches found” but I went into the darktable cache directory and manually deleted everything. Upon retsrarting sam21 works again perfectly. As wrote earlier it seems better than AI selections in paid apps I own or haven’t yet finished my subscription.
Is there a way to clear the darktable caches in the GUI? That might be helpful to newcomers who run into the same issue I had with sam2.1.