Working with modules from Lua scripts in darktable

priort · February 25, 2023, 4:08am

I think you will see it comes from added exposure and the clarity preset… I found a small typo in the code…after that things seem to work properly for me…

EDIT @Ulrich_Gesing

This is the bit…note the labels for the options… the values for setting the module option were also set like this for the function… I have changed the comma out for a period and it seems to work… that typo seems to have caused the weird errors I saw in 4.21. I didn’t have the lua logging on to confirm but that could confirm…

function StepContrastEqualizer:Init()
self.ComboBoxValues =
{
“unchanged”,
“reset to default”,
“clarity, strength 0,25”,
“clarity, strength 0,50”,
“denoise & sharpen, strength 0,25”,
“denoise & sharpen, strength 0,50”
}

self.Widget = dt.new_widget(“combobox”)
{
changed_callback = ComboBoxChangedCallback,
label = “contrast equalizer”,
tooltip = self.Tooltip,
table.unpack(self.ComboBoxValues)
}
end

function StepContrastEqualizer:Run()
local selection = self.Widget.value

if (selection == “unchanged”) then
return
end

self:LogStepMessage()
self:EnableDarkroomModule(“iop/atrous”)
GuiActionWaitForPixelPipe(“iop/atrous”, 0, “reset”, “”, 1.0)

if (selection == “clarity, strength 0,25”) then
GuiActionWaitForPixelPipe(“iop/atrous/preset/clarity”, 0, “button”, “on”, 1.0)
GuiActionWaitForPixelPipe(“iop/atrous/mix”, 0, “value”, “set”, “0.25”)
end

if (selection == “clarity, strength 0,50”) then
GuiActionWaitForPixelPipe(“iop/atrous/preset/clarity”, 0, “button”, “on”, 1.0)
GuiActionWaitForPixelPipe(“iop/atrous/mix”, 0, “value”, “set”, “0.5”)
end

if (selection == “denoise & sharpen, strength 0,25”) then
GuiActionWaitForPixelPipe(“iop/atrous/preset/denoise & sharpen”, 0, “button”, “on”, 1.0)
GuiActionWaitForPixelPipe(“iop/atrous/mix”, 0, “value”, “set”, “0.25”)
end

if (selection == “denoise & sharpen, strength 0,50”) then
GuiActionWaitForPixelPipe(“iop/atrous/preset/denoise & sharpen”, 0, “button”, “on”, 1.0)
GuiActionWaitForPixelPipe(“iop/atrous/mix”, 0, “value”, “set”, “0.5”)
end
end

priort · February 25, 2023, 4:25am

If you haven’t changed it then look at the color balance module. Do you have 15 and 25 as the settings for chroma and sat and if you go to the mask tab are the fulcrum settings changed… if not then you have the same baseline as it has in 4.21. The changes you see are from auto exposure and the full clarity setting in the contrast eq which is quite strong. Correcting the typo in the script gives the full functioning of the macro. If you can confirm that then we know it works in WIndows up to your version but gets broken between 4.3 _705 and +750…

Bernhard_Vogler · February 25, 2023, 5:37am

This is what I see:

One thing that is funny, though, that when I try to run a second set of pictures it will not do anything. If I quit and restart darktable it then will do. It appears it works only on the first execution

priort · February 25, 2023, 5:49am

Not sure where that comes from… run it on a duplicate…??? Ah your adding one of the presets…I was just setting the mask tab option…

In any case it didn’t work fully until I made the edit to the script… And it works in 4.21 fully now but not in my current version…

priort · February 25, 2023, 6:05am

Okay I think now maybe it a language/local setting… I am in Canada and so 0,25 was not correctly interpreted but maybe its not a typo and gets used correctly in the region(s) where the settings would accept a comma as a decimal…

Ulrich_Gesing · February 25, 2023, 10:38am

Thank you for your research! The analysis sounds plausible, here with my German locale setting it works. I will adjust it.

Sciencenerd · February 25, 2023, 1:41pm

Thanks, Uli, for all your hard work on this. I can confirm Todd’s finding that changing 0,25 and 0,50 to 0.25 and 0.50 fixes the hang at the contrast equalizer stage (I’m in the US, that’s probably why).

However, for me, the script still exhibits the same problem as before namely that is sometimes processes images multiple times, failing to move on to the next image. Sometimes it moves on, sometimes it doesn’t, it seems random. I know there was a discussion about timing of events etc. not sure what the conclusion was on this.

BTW: I am using darktable 4.2.0 on Win 10.

priort · February 25, 2023, 1:48pm

It a great script. Now I just have to see if I can find where it got broken along the line to the current master… There were a couple of changes made to model naming. It seems to run throught the script and activate all the specified modules but none of the setting are applied… maybe it’s doing the reset part but not continuing??

Ulrich_Gesing · February 25, 2023, 2:12pm

I can reproduce it now: After each GuiAction, the script waits for a “pixelpipe-processing-complete” event. If an exception occurs, the event remains registered. Next execution/event registration fails and the script does nothing until you restart Darktable. I fixed that. An additional “destroy_event” helps.

Ulrich_Gesing · February 25, 2023, 2:29pm

I could reproduce it after switching my desktop to “Canadian”. There is a simple solution: I replaced the string parameters “0,25” with their decimal equivalents 0.25. This avoids the conversion. Attached you can find the new version.

InitialWorkflowModule.zip (10.4 KB)

Ulrich_Gesing · February 25, 2023, 2:51pm

Right now, I cannot reproduce that. Could you explain, how you do it? How do you select the images?

Could you send me a logfile, as described in darktable lua documentation - troubleshooting

priort · February 25, 2023, 3:26pm

log.txt (6.7 KB)
log2.txt (6.1 KB)
log3.txt (5.4 KB)
log4.txt (5.6 KB)

Here are a few log files… after looking at the first one I though thought the raw CA was the issue so I set it to unchanged…things seemed to go a bit further but a similar error would pop up…

So this is the behaviour from the current master… maybe a little behind now a day or so… running Win11

4.3 + 750…

Ulrich_Gesing · February 25, 2023, 3:44pm

I will take a look at this.

@Claes Thank you for your howto “How to get the master branch compiled on my local EndeavourOs machine” (with my brand new github account :-))

Ulrich_Gesing · February 25, 2023, 4:08pm

With 4.3.0+801: The script checks if the module is already enabled before activating it:

local status = dt.gui.action(“iop/cacorrect”, 0, “enable”, “”, nil)

That does not work anymore, it is the same for other modules:
LUA ERROR : bad argument #5 to ‘?’ (number expected, got nil)

Apparently the API has changed? How can you read this value now?

priort · February 25, 2023, 4:44pm

@dterrahe @wpferguson One of you two might know??

priort · February 25, 2023, 4:52pm

Maybe answer lies in here…

github.com/darktable-org/darktable

define widget type mimicking actions with flexible lua scripts

darktable-org:master ← dterrahe:lua_process_action

opened 11:27PM - 08 Feb 23 UTC

dterrahe

+853 -723

This PR has picked up a bunch of separate functionalities with interdependencies… (so splitting into separate PRs would involve extra work, waiting and rebasing). It might be best just to list some release notes entries: - the lua call to `dt.gui.action` has become more flexible, with most parameters optional, so you can read the focused status of a module doing just `dt.gui.action("iop/filmicrgb", "focus")` - tooltips show these compacter lua commands in mapping mode (only adding the last parameter, _instance_, if the module supports multi-instance) and have been added to presets and styles menus as well - lua commands can be copied to the clipboard, using ctrl+v in the shortcuts dialog, both from a selected action or shortcut, or long right click when in mapping mode (over a widget) or in the presets/styles menus - the shown/copied lua command for a slider or combobox will set the value it currently has - a bug with shortcuts (and `dt.gui.action`) to set the active item in a combobox with varying content has been fixed and now it is also possible to directly set the values of the combos for the focused module's blending mode etc. (by setting the shortcuts _effect_) - added section headers to the _sort by_ drop-down (_files_, _times_, etc) - shortcuts assigned to presets or styles will be shown when hovering over them in their menu - _long_ left clicking a preset will keep the menu open so you can quickly switch between several to see the effect without having to repeatedly click the preset button to reopen the menu. You can also scroll over the preset button to switch to previous/next presets (like you already could via shortcuts) - a shortcut can now be directed to a lua script that mimics a standard slider, dropdown or button, but dynamically selects the real widget(s) that receive(s) it, based on for example which module is focused or enabled. The advantage is that all fallbacks work as normal, so you can assign a midi knob to it and turning it (holding shift/ctrl to speed up/down) or pressing it to reset work regardless of which widget receives it. Basically this is a much more flexible alternative to the fake widgets under `processing modules/<focused>`. This allows owners of for example an x-touch mini to use their scarce rotors in different, fully configurable, ways while working in different modules (which can also be focused using midi buttons which will then light up). Such configurations could be shared using https://github.com/darktable-org/lua-scripts - support shortcuts to sliders/combos created in lua, either via visual mapping mode or in the shortcuts dialog under the _lua_ category. _Elements_ and _effects_ are not supported. ORIGINAL POST: Introduce a new lua call `dt.gui.mimic` that creates an action for a "fake" widget (in the shortcuts action list under "lua scripts"). The widget _looks_ like a normal _slider_ or _dropdown_, so when a shortcut is mapped to it, it accepts the same elements and effects and uses the same fallbacks. But instead of sending these directly to a _real_ widget, it calls a lua routine, which can then decide (based on for example which module has the focus, but this could be anything) to which _real_ widget the shortcut should be forwarded. It does this by calling `dt.gui.action` with the arguments it received. @jenshannoschwalm You could do what you mentioned in https://github.com/darktable-org/darktable/pull/11361#issuecomment-1152726578 but instead of having different modifiers direct where the _contrast_ knob goes, it could depend on whether filmic is currently focused. And shift and ctrl would still work to speed it up or slow it down. ``` local dt = require "darktable" dt.gui.mimic("slider", "multi-contrast", function(action, element, effect, size) if dt.gui.action("iop/filmicrgb", "focus") == 1 then which = "iop/filmicrgb/contrast" else which = "iop/colorbalancergb/contrast" end return dt.gui.action(which, element, effect, size) end ) ``` You could also have one shortcut be sent to two widgets at the same time ``` dt.gui.mimic("dropdown", "double combo", function(action, element, effect, size) dt.gui.action("iop/filmicrgb/contrast in shadows", element, effect, size) return dt.gui.action("iop/filmicrgb/contrast in highlights", element, effect, size) end ) ``` In this case, if mapped to a knob on the x-touch mini, the rotator lights would indicate the status of the second widget. This could be expanded, for the x-touch or similar, to define 8 fake sliders and assign all of the rotator knobs to those. They could then be redirected to the different point on the curve in color zones, if that module is active, or to the different parameters of shapes, if a mask is currently being edited (test `dt.gui.action("iop/blend/tools/show and edit mask elements", 0, "", "")`), and to some other frequently used sliders when neither of these are the case. Old style calls are still supported so existing scripts shouldn't break. EDIT: makes calls to dt.gui.action more flexible and moves _instance_ parameter to the end (since it is only relevant for a subset of actions (multi-instance iop modules) and even then usually defaults to 0. Most parameters are now optional, so you can read the focus state of a module with `dt.action.gui("iop/modulename", "focus")` or set it with `dt.action.gui("iop/modulename", "focus", 1)` EDIT2: fixes a bug with combobox shortcuts/dt.gui.action calls that specify a specific "item:" in cases where the list in the live combo options was manipulated at runtime. The nth item in the combo would not correspond to the nth item in the actions list anymore. Now more robust. Could be used to provide a static list of options in, say, the blend mode dropdown as well to allow them to be selected by shortcut.

Ulrich_Gesing · February 25, 2023, 5:07pm

that’s it!

I removed the last two parameters from
local status = dt.gui.action(“iop/cacorrect”, 0, “enable”, “”, nil)

after that it works, but take a look at the following part of the discussion. It should work with both versions:

“I have now made this change and made all except the first argument (the action itself) optional while still supporting the previous order (as long as in that case all parameters are specified, which used to be a requirement).”

Sciencenerd · February 25, 2023, 7:04pm

log.txt (57.0 KB)

Here is a sample log file. I selected 9 images in the lighttable to process, then started the script. after processing the first image, it moved on to the next image in the filmstrip, but the main image window stayed on image number 1 and reprocessed it. This repeated until it finally loaded image 8 and processed it twice. So, from 9 selected images, only two got processed, as I can also see in the .xmp files.

Funny enough, on my work laptop (same darktable version, but Win11), it seems to work perfectly fine.

dterrahe · February 25, 2023, 7:22pm

I’m sorry, I didn’t realise people used nil to indicate NAN as the move_size parameter. So I only tested if “” and 0/0 worked. Anyway, I’d recommend moving to the new “format” as you probably barely are using the “instance” parameter. So you can just use
local status = dt.gui.action(“iop/cacorrect”, “enable”)

Ulrich_Gesing · February 25, 2023, 8:25pm

Sounds like a timing problem.

Perhaps the event, the script is waiting for, should come later. Perhaps the loading process is currently being aborted and the processing of the image is started too early?

Is your working PC significantly faster?

I will try to reproduce it, maybe a slow virtual machine can help.