Bug (?) in new ai masking tool in darktable 5.5

Software darktable
, ,
1

I have been playing with the new ai subject selection tooling in the 5.5 darktable build (I am on Mac - thanks @MStraeten for the build!)

It all worked like a charm! No problem with downloading the models, the button for create a mask is a logical place it all works very nicely.

I was able to create this mask. Using the instructions at the top for adding to the mask, clear something from the mask. Really works well!

mask created
mask created1710×1154 187 KB

But in this photo, the small branch on the tree at right left of the corner of the frame is also part of the foreground.

So when trying to add to the mask this is the result:

only the small branch selected
only the small branch selected1708×1178 197 KB

Although I added to the mask (with clicking on the branch) the mask is now on that small branch.

Is this is a bug? or is it it a limitation of the solution that a mask must continuous?

@anry Thanks for the very nice work on this tool!

2

Did you try Ctrl+Click? On the tooltip it says that you can use that to add to the mask.

Also, when you are happy with selection you have to right click outside the mask to actually fix the path. This was not obvious to me from the UI.

Since each mask becomes a path, you can also create a separate mask for the spurious branch. It will just become another path.

I hope what I am saying is all correct, I just started playing with it myself :slight_smile:

1 Like
3

instructions
instructions918×130 11.4 KB

These are instructions that I followed. And to create that mask in the first picture is started was already with multiple 'click’s. It was then I clicked the small branch the mask changed unexpectedly.

4

It’s not a bug, and I wouldn’t call it limitation as well. You just need to keep in mind that fundamentally it is a single object selection tool. This object must be visually continuos.

In your case, even if we know that small branch is a part of the tree, it is visually separate. So it is not possible to select both of them in one try. But that is not a problem actually. You just run mask tool one more time and group results.

5 Likes
5

Sure it is not a problem and make a group. I was just not if this was the expected behaviour.

2 Likes
6

Sure. Thank you for testing. That was actually very reasonable and popular question. I would reflect this in the docs.

3 Likes
7

Great! Thanks for developing this!

3 Likes
8

Thank you for your time and the effort you’ve put into the module.

When the AI function is activated, the image is ‘processed’; this currently runs on the CPU, with MIGraphX selected in the settings. The latest ROCm version also runs without any issues.
MIGraphX is also installed.

Edit: Arch Linux, ryzen ai 395 max with 8060S iGPU

1 Like
9

I’m interested in trying this but I can’t find where to use it in 5.5, where can I find more information about it?

10

@anry …
Hi just for the sake of interest , why have you chosen the specific SAM2 model ?
SAM 2 offers other models … are there any differenecs between the various models ?

Cheers Andreas

11

We currently have two segmentation architectures available: SAM 2.1 and SegNext.

SAM 2.1 comes in four sizes (Tiny, Small, Base Plus, Large) that share the same architecture but differ in encoder capacity - larger models produce more accurate masks but are slower. We included Tiny, Small, and Base Plus. Tiny and Base Plus can be manually downloaded from the repository and installed in AI preferences tab. We skipped Large for now since its encoder is ~900 MB and too heavy for interactive use.

12

I will make a separate post about how to use AI object masking tool. Meanwhile, you can either download nightly build for Windows or macOS, or build darktable with --enable-ai option. After that you need to enable AI features in preferences and download AI models.

13

@anry … i would rather like to have a more detailed mask , that does take longer to be processed , than a quicker less " precise " mask :grinning:

But will try to get the base plus model … hopefully i am not too dumb , to get it downloaded and installed .

14

I had a look … but could only find " tiny " and " small " !

15

Small variant is available in AI preference tab by default.
Tiny can be downloaded manually from Release v5.5.0.4 · darktable-org/darktable-ai · GitHub
Base plus variant will be available in the mentioned repository soon.

16

Cool anry … will wait for Base plus
THX again for your effort !!!

1 Like
17

Would it be performant on a large graphics card?