Python plug-in: run SMART (AI mask maker based on SAM2)

Software GIMP
1

I have written new mask making plug-in for GIMP3. @agriggio made AI mask making program using META’s SAM2 model named “SMART”. His program was written for ART, but it is not usable (and useful) only for ART. My plug-in executes SMART from GIMP3, and imports the result mask image as layer mask of GIMP. This is a trial version and is only executable on Windows, for now.

INSTALATION PROCESS

  1. Install SMART
    Download SMART from following link, and install it in “C:\Program Files\SMART” .
    Release v0.2 · artraweditor/SMART · GitHub

  2. Download my plug-in from following link and install it under plug-in directory of GIMP3.
    https://drive.google.com/file/d/1HyX3sBq62bx-do7sVgPLhSNcmMGN4gpU/view?usp=drive_link

  3. Download model checkpoint
    From following link, download “sam2.1_hiera_base_plus.pt”, and, put it in the “models” directory under “C:\Program Files\SMART\” .

USAGE

  1. Run this plug-in from “menu > layer > mask > Parametric mask > run SMART”.

    image
    image1029×800 206 KB

  2. Make the mask using shift + left click.

    image
    image1274×936 243 KB

  3. Save the mask image with default file name (Do not change the file name!).

    image
    image1225×869 252 KB

  4. Exit SMART.

  5. The mask image file (_tmp_mask.png) will be imported in GIMP as layer mask.

    image
    image478×229 13.7 KB

3 Likes
2

Not sure if you can help. I downloaded the zip file for SMART, I extracted the file and then created a new folder in program called SMART. I pasted the the contents for Smart as shown in the first screenshot here. Then I installed the plugin for GIMP. When I try to run SMART I get the following error. Thanks in advance

image
image438×313 7.39 KB

image
image856×282 64.3 KB

3

Sorry, I forgot to include some of the instructions.

From following link, download “sam2.1_hiera_base_plus.pt”

And, put it in the “models” directory under “C:\Program Files\SMART\”

4

image
image420×249 10.9 KB

Why not download all four models?

5

Because SMART.exe only uses it.
Originally SMART is python scripts. However Alberto convert them to a binary file for installation convenience. So you cannot edit checkpoints model setting and you cannot use the other checkpoints models on Windows.

6

Thanks for your reply. I will let you know how I get on.

7

Proof that it worked. Not perfect but not bad for my first attempt.

image
image766×1116 409 KB

1 Like
8

Thank you very much for all the previous ones as well as this plug-in.
Installed running, trained works properly.
But with my not-too-powerful Laptop AMD Ryzen, with virtually no GPU, operations last quite slowly.
I am frankly shocked that news authors do not pay attention to future stakeholders,
That AI technology requires computers with powerful GPUs.
Therefore, many beginners are often disappointed with the results they receive , or lack thereof.

PS
Information about all published plug-ins, have been developed and published (PL) stating clearly who the author is

Greetings

1 Like
9

I also use not-powerful Laptop (RAM: 8G, storage: SSD, no GPU, CPU: intel 6th gen., which cannot be upgraded to Win 11), but it is not unbearably slow. If you use HDD as storage, it’s worth considering replacing it with a SSD.

10

Hi,
I release complete version (ver. 0.2) of this plug-in.

INSTALATION PROCESS for Linux and Mac OS

  1. Install SMART
    Install SMART following Alberto’s instructions in below link.
    ART and AI-based tools | ART raw image processor
    This instructions premise the SMART installed directory as “$HOME/src/SMART”, the Python vertual environment directory as “$HOME/src/ART-AI-venv” and the path of Pyhon interpreter as “$HOME/src/ART-AI-venv/bin/python”.
    If you change these directories, please adjust the path name accordingly.

  2. Download my plug-in from following link and install it under plug-in directory of GIMP3 and make it executable.
    https://drive.google.com/file/d/1o-YfsPDD6iCIuarTjvDAW3pDr8HK5WDP/view?usp=drive_link

  3. Download model checkpoints
    From following link, download “sam2.1_hiera_base_plus.pt”, and, put it in the “models” directory under the SMART installed directory. You can use other models, however it needs to adjust the SMART scripts.
    GitHub - facebookresearch/sam2: The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

  4. Adjust the path of my plug-in
    Adjust the path name of following parts in my plug-in to the appropriate name according of your environment.

elif platform.system() == "Linux":
    command = "/home/%USERNAME%/src/ART-AI-venv/bin/python"
    command1 = "/home/%USERNAME%/src/SMART/src/main.py"
elif platform.system() == "Darwin": #Mac OS
    command = "/Users/%USERNAME%/src/ART-AI-venv/bin/python"
    command1 = "/Users/%USERNAME%/src/SMART/src/main.py"
  1. If you use the Flatpak version of GIMP3 on Linux, use the following command to run the GIMP3.
flatpak run --socket=session-bus org.gimp.GIMP
--verbose

ATTENTION FOR SMART INSTALLATION

In some Linux distro, you may encounter troubles in installation of wxPython. And if your PC doesn’t have NVIDIA or AMD GPU, you may have trouble in installation of SAM2.

To solve these problems, please refer my posts in the link below.