Plugin for creating automatic tags

scheckmedia · May 12, 2020, 5:59pm

Hello to every one,

I’m Tobi, new in this community and four weeks ago I switched from Lightroom to darktable. There are multiple reasons but I would say the main reason is that darktable is Open Source and runs on Linux.

I’m very happy with darktable and thanks to the great videos of @Bruce_Williams I could find my way around very quickly.

Anyway, mainly I work as researcher/developer and because of this, I started working on an auto-tagging plugin in Lua for darktable, which I would like to present here to get feedback. It uses a small command-line application, called photils-cli, to extract features from an image. These features will be used to detect keywords/tags (at no time your image leaves your device, only a representation).

I would be very happy if it is tested by some of you and you could give me some feedback.

Many thanks and best regards

Tobi

rgo · May 12, 2020, 7:09pm

It’s great, I’ll try it this night.

Thanks for sharing!

BTW I didn’t know about photils but I like this “… The usage of the feature vector has this advantage that at no time your selected image will leave the PC.”
From privacy-wise is really good and for bandwith too.

durian · May 12, 2020, 7:56pm

I tried to install it, but the instructions are not clear to me. Where should I put the photils AppImage? It needs to be found by the lua plugin I guess? The darktable lua instructions were not clear. I did put the lua script in the darktable lua/contrib directory, and added ‘require “contrib/photils”’ to the luarc file. But nothing shows up in darktable under tagging.

-peter

Tobias · May 13, 2020, 8:34am

@scheckmedia Is it possible to setup my on server, or do everything local?

anon41087856 · May 13, 2020, 8:46am

I remember someone did a similar Lua script for face recognition + tagging with Tensor Flow. I wonder if we shouldn’t ship these officially at some point, at the condition that the machine learning is done locally and not outsourced to a server/third party.

@scheckmedia That’s awesome. Would it be possible to include support for hierarchical tags ?

scheckmedia · May 13, 2020, 8:49am

I think I can implement something in the photils-cli application but this will blow up the file size to ~500 MB. The reason is that I have to provide the reference features and the corresponding tags what causes the file size.

From that side the hosted approach is more resource friendly. But sure I can think about some optional flag to provide an offline option.

scheckmedia · May 13, 2020, 8:54am

I think this is possible as well. If someone can provide me some hierarchical structure I can improve the tagging in the near future.

I should mention that the machine learning part is done locally! It uses TensorFlow lite for the feature extraction. Only the request to find images and tags with the same feature of a given image is done online.

scheckmedia · May 13, 2020, 8:58am

Thank you for your feedback. I will update the installation instructions and extend it with a short video. I’ll update the repository later today.

Maurizio_Paglia · May 13, 2020, 9:31am

This is veri interesting!
I will surely try it asap.
Thanks!

Tobias · May 13, 2020, 9:44am

@anon41087856 We have the face recognition Lua script in the official Lua github repository here:

github.com

darktable-org/lua-scripts/blob/master/contrib/face_recognition.lua

--[[
  Face recognition for darktable

  Copyright (c) 2017  Sebastian Witt
   
  darktable is free software: you can redistribute it and/or modify
  it under the terms of the GNU General Public License as published by
  the Free Software Foundation, either version 3 of the License, or
  (at your option) any later version.

  darktable is distributed in the hope that it will be useful,
  but WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  GNU General Public License for more details.

  You should have received a copy of the GNU General Public License
  along with darktable.  If not, see <http://www.gnu.org/licenses/>.
]]

--[[

This file has been truncated. show original

And it uses this program for face recognition:

Marcsitkin · May 13, 2020, 12:25pm

Looks like a useful script, but I’ve tried to install it and the get tags panel doesn’t appear after copying the two files to the suggested folder in the ~/.confi/darktable directory. Hopefully you can clarify the installation. Pretty sure lua is working, I use the export to GIMP script frequently.

Marcsitkin · May 13, 2020, 12:50pm

Update: have installed the lua and linked it to the executable, but nothing happens when I selec an image and hit “Get Tags” . Is the location of the AppImage important?

scheckmedia · May 13, 2020, 1:31pm

Meanwhile, I’ve uploaded a video with the whole installation process: https://photils.app/media/installation.mp4

Hope this can clarify some issues.

paperdigits · May 13, 2020, 2:58pm

Welcome @scheckmedia! This is quite the first contribution.

GLLM · May 13, 2020, 9:46pm

Hi @scheckmedia !

Thanks for your script, which looks really nice !!
I installed it without issues, but I have an issue when running it … it seems the script is allergic to filepathes containing spaces !
Do you confirm ?

Do you want/need me to book an issue on GH ?
Thanks again !
M_LL_G

Tobias · May 14, 2020, 7:18am

To handle spaces in file names you could use the
sanitize_filename function:

github.com

darktable-org/lua-scripts/blob/master/lib/dtutils/file.lua#L557

    
      
          --    result = os.execute('if exist "'..filepath..'" (cmd /c exit 0) else (cmd /c exit 1)')
          --    if not result then
          --     result = false
          --    end
            elseif (dt.configuration.running_os == "linux") then
              result = os.execute('test -e ' .. dtutils_file.sanitize_filename(filepath))
              if not result then
                result = false
              end
            else
              local file = io.open(filepath, "r")
              if file then
                result = true
                file:close()
              else
                result = false
              end
            end
          
          
  return result
          end

scheckmedia · May 14, 2020, 7:26am

Thanks for this, meanwhile I solved the problem and I will update the script. I’ve not used quotation marks while I passed arguments to photils-cli. This works until spaces appear.

Tobias · May 14, 2020, 7:33am

The problem is, it works different on windows and Linux. @wpferguson tested a lot to find all the nasty details.

scheckmedia · May 14, 2020, 7:55am

Ok, this explains it. I’ve changed the script to use sanitize_filename for filename arguments.

gadolf · May 14, 2020, 12:02pm

Thanks for the work.
I’m totally new to neural networks, so how does it learn and show more specific tags the next run? From my local tags?
I selected an image from Sugar Loaf mountain, a more a less iconic mountain, and it didn’t show any tag related to this country (Brazil).
Here’s the image: