Working with modules from Lua scripts in darktable

Software darktable
, ,
9

Apologies in advance. This is going to be long, but I’ll break it into sections so you can jump to what interests you.

Preface

A couple of years ago I started thinking about “smart” styles; i.e. styles, or parts of styles that could be applied based on the image. I didn’t really know how to do it, but the idea was simmering in the recesses of my brain waiting for a solution.

Then…

@dterrahe rewrote darktable’s input system to accommodate controllers other than keyboards and mice. While he was doing that he came up with a way to expose the input system to lua, thus dt.gui.action() was born. At the time he wrote it I wasn’t really sure how or what to use it for.

Then…

A couple of months ago I was editing a bunch (100+) images that had blown specular highlights (lights reflecting off of helmets). My fix, to get rid of the ugly flat blue area caused by filmic chrominance preservation being set to max rgb was to open filmic and set the chrominance preservation to luminance y then open the highlight reconstruction module and set the mode to segmentation based.

So, to fix the highlights I would click to open filmic, click to choose the options tab, click to drop down the chrominance preseveration list, click to select luminance y, click to open the highlight reconstruction module, click to drop down the mode list, and click to select segmentation based.

Each image I was spending 45 seconds to a minute just clicking around the UI, while visions of a carpal tunnel brace danced through my head. About halfway through, as I’m thinking there has to be a better/smarter way to do this, I thought of dt.gui.action().

Early Steps

The first GUI automation I attempted was blown highlights.

This is the result

local dt = require "darktable"

local mods = {}

mods.colorbalancergb = "iop/colorbalancergb"
mods.color_calibration = "iop/channelmixerrgb"
mods.diffuse_or_sharpen = "iop/diffuse"
mods.denoise_profiled = "iop/denoiseprofile"
mods.exposure = "iop/exposure"
mods.filmicrgb = "iop/filmicrgb"
mods.highlight_reconstruction = "iop/highlights"

local function select(mod, item, instance, value)
  local inst = instance or 0
  dt.gui.action(mod .. "/" .. item, inst, "selection", value, 1.0)
end

dt.register_event("blown_highlights", "shortcut",
  function(event, shortcut)
    select(mods.filmicrgb, "preserve chrominance", 0, "item:luminance Y")
    select(mods.highlight_reconstruction, "method", 0, "item:segmentation based")
  end , "blown highlights"
)

I assigned the keystroke alt+b to the shortcut. I used the shortcut and in less than a second I had performed the same function that took 45+ seconds and 7 clicks.

Next I went for a bigger step. I like my verticals to be vertical. I use the rotate and perspective module then right click and draw a line on something vertical to straighten my image. I had a thought that if the image had enough detectable vertical lines, then the rotate and perspective module could be automated to straighten the image.

Developing this is where we ran into bugs that caused some sudden and violent darktable crashes. @dterrahe found and fixed the bugs, and IIRC the fixes are in darktable 4.0.1.

The code looks like this:

local dt = require "darktable"

local function button(mod, item, instance)
  local inst = instance or 0
  dt.gui.action(mod .. "/" .. item, inst, "", "", 1.0)
end

dt.register_event("auto_straighten", "shortcut",
  function(event, shortcut)
    dt.gui.action("iop/ashift", 0, "focus", "on", 1.0)
    dt.control.sleep(500)
    button("iop/ashift", "structure/auto", 0)
    dt.control.sleep(1500)
    dt.gui.action("iop/ashift/fit/vertical", 0, "", "", 1.0)
    dt.control.sleep(500)
    dt.gui.action("iop/ashift", 0, "focus", "off", 1.0)
    --button("iop/ashift", "structure/auto", 0)
  end , "auto straighten"
)

Again I assigned the keystroke alt+s to this and in a couple of seconds it will automatically straighten an image if there are enough detectable vertical lines. If not, it distorts the image and you just have to reset the module and do the right click method.

In this module I had to use 1/2 second sleeps to allow the module time to process the image, otherwise the line detection didn’t complete and the result was garbage.

Here’s a picture that has enough verticals to work

100R7LR3_3J6A0329.cr3 (34.9 MB)

This file is licensed Creative Commons — Attribution-ShareAlike 4.0 International — CC BY-SA 4.0

Load the script, assign a shortcut, open it in darkroom view, make sure rotate and perspective is visible in your module list, activate the shortcut, watch the image get straightened.

I have a style that I apply that has the modules I use, in the order I want them, with most of them turned off due to the amount of CPU/GPU they require (i.e. multiple instances of diffuse and sharpen). I adjust exposure, rotate and crop, then activate the modules that are turned off to complete the image. I shoot sports, often in bad light, so depending on ISO different modules are turned on. I set up several different shortcuts to handle automatically turning on those modules just using a shortcut. It took @dterrahe answering MANY questions until I learned to traverse the history stack and turn on the correct modules.

Smarter Steps

When applying my final modules I would look at the ISO of the image and then trigger the appropriate shortcut to turn on the correct modules. Between fat fingers and sometimes forgetting to check the ISO I decided the next target was to add some “smarts”.

The ISO is accessible for each image, so my current version of enable final modules checks the ISO of the image being edited and activates the correct set of modules.

Next Steps

In order to have “smarts” we have to have image information. darktable has a module that will show me if an image is over/under exposed, but it only provides that information in a visual format. I need to be able to query the function and get an response back that can be evaluated by a program. So for instance the script could query and get back over exposed, 30% and then the program could change the filmic chrominance preservation and select the appropriate highlight reconstruction method (probably segmentation based since it’s a large area).

The more we try to automate, the more we can figure out what information needs to be exposed to lua so that we can use it to make smarter scripts.

@donda as I doing the first automation I thought about the timelapse crowd. In the goodies below there is a shortcut to look at the current exposure level, then adjust it by some EV. Unfortunately dt.gui.action() works only on the image in the darkroom view when adjusting exposure. But, there are functions available from lua, dt.gui.views.darkroom.display_image(), that loads an image into darkroom view for editing so you could select a set of images then have a script load each one into darkroom view and adjust the exposure. This might be another of the items where you need more information than what’s currently available to make the correct adjustment.

To Do

  1. Figure out how to make this accessible to the masses. If you can code, you can automate you workflow and achieve some remarkable speedups. I used to spend 3-5 minutes per image and now my workflow is ~30 seconds per image. If you can’t code then you are out in the cold. I’ll start a library of usable functions, but we need to figure out a better interface than just shortcuts. And we need to figure out a way that a user can set up an automation (maybe something similar to a macro recorder?).

  2. Figure out how and what information we can expose to lua so we can make smarter scripts.

  3. Figure out how to document this function. There are so many different ways to use it and do things but describing them isn’t going to fit in a page or two in a manual.

Goodies

I’ve talked enough so here’s some toys to play with. One is my starting point style that the scripts are designed to work with and the second is the script itself. It should work on darktable 4.0.1, but will definitely work on master.

The script is a snapshot of my work in progress. There are several useful shortcuts in there and some code that should be deleted, but is still hanging around. Of particular note is the last function which creates the starting point style all in the darkroom gui by activating modules, putting them in the correct place, then turning them off. It’s slower that just applying a style since it has to manipulate the GUI, but you can see how it’s done.

Starting Point 1.dtstyle (4.9 KB)

smart_styles.zip (2.1 KB)

Final Notes

Thank you @dterrahe for creating this and educating me in it’s use.

28 Likes