Plugin for creating automatic tags

Hello to every one,

I’m Tobi, new in this community and four weeks ago I switched from Lightroom to darktable. There are multiple reasons but I would say the main reason is that darktable is Open Source and runs on Linux.

I’m very happy with darktable and thanks to the great videos of @Bruce_Williams I could find my way around very quickly.

Anyway, mainly I work as researcher/developer and because of this, I started working on an auto-tagging plugin in Lua for darktable, which I would like to present here to get feedback. It uses a small command-line application, called photils-cli, to extract features from an image. These features will be used to detect keywords/tags (at no time your image leaves your device, only a representation).

I would be very happy if it is tested by some of you and you could give me some feedback.

Many thanks and best regards

Tobi

10 Likes

It’s great, I’ll try it this night.

Thanks for sharing! :slight_smile:

BTW I didn’t know about photils but I like this “… The usage of the feature vector has this advantage that at no time your selected image will leave the PC.”
From privacy-wise is really good and for bandwith too.

I tried to install it, but the instructions are not clear to me. Where should I put the photils AppImage? It needs to be found by the lua plugin I guess? The darktable lua instructions were not clear. I did put the lua script in the darktable lua/contrib directory, and added ‘require “contrib/photils”’ to the luarc file. But nothing shows up in darktable under tagging.

-peter

@scheckmedia Is it possible to setup my on server, or do everything local?

I remember someone did a similar Lua script for face recognition + tagging with Tensor Flow. I wonder if we shouldn’t ship these officially at some point, at the condition that the machine learning is done locally and not outsourced to a server/third party.

@scheckmedia That’s awesome. Would it be possible to include support for hierarchical tags ?

1 Like

I think I can implement something in the photils-cli application but this will blow up the file size to ~500 MB. The reason is that I have to provide the reference features and the corresponding tags what causes the file size.

From that side the hosted approach is more resource friendly. But sure I can think about some optional flag to provide an offline option.

I think this is possible as well. If someone can provide me some hierarchical structure I can improve the tagging in the near future.

I should mention that the machine learning part is done locally! It uses TensorFlow lite for the feature extraction. Only the request to find images and tags with the same feature of a given image is done online.

Thank you for your feedback. I will update the installation instructions and extend it with a short video. I’ll update the repository later today.

1 Like

This is veri interesting!
I will surely try it asap.
Thanks!

@aurelienpierre We have the face recognition Lua script in the official Lua github repository here:

And it uses this program for face recognition:

1 Like

Looks like a useful script, but I’ve tried to install it and the get tags panel doesn’t appear after copying the two files to the suggested folder in the ~/.confi/darktable directory. Hopefully you can clarify the installation. Pretty sure lua is working, I use the export to GIMP script frequently.

Update: have installed the lua and linked it to the executable, but nothing happens when I selec an image and hit “Get Tags” . Is the location of the AppImage important?

Meanwhile, I’ve uploaded a video with the whole installation process: https://photils.app/media/installation.mp4

Hope this can clarify some issues.

1 Like

Welcome @scheckmedia! This is quite the first contribution. :slight_smile:

1 Like

Hi @scheckmedia !

Thanks for your script, which looks really nice !!
I installed it without issues, but I have an issue when running it … it seems the script is allergic to filepathes containing spaces !
Do you confirm ?

Do you want/need me to book an issue on GH ?
Thanks again !
M_LL_G

To handle spaces in file names you could use the
sanitize_filename function:

Thanks for this, meanwhile I solved the problem and I will update the script. I’ve not used quotation marks while I passed arguments to photils-cli. This works until spaces appear.

The problem is, it works different on windows and Linux. @wpferguson tested a lot to find all the nasty details.

1 Like

Ok, this explains it. I’ve changed the script to use sanitize_filename for filename arguments.

Thanks for the work.
I’m totally new to neural networks, so how does it learn and show more specific tags the next run? From my local tags?
I selected an image from Sugar Loaf mountain, a more a less iconic mountain, and it didn’t show any tag related to this country (Brazil).
Here’s the image: