Darktable performance in lighttable mode (tagging etc)

Software darktable
1

I find that Darktable is a bit slow and laggy when tagging images and setting star ratings or color labels.

I’m using keyboard shortcuts to do this, and I’ve tried to work on local copies too; usually my raws sit on an external hd, but as a test I decided to move a set of photos on the internal ssd to see if this could help. I understand that all these actions (tagging, rating, etc) work on the sidecar files – or is it on library.db? Anyway, I’m not seeing any major differences also when working on local copies.

I’ll add that the machine in use should be powerful enough for this (dell xps15 with core i7@2.80Ghz, geforce gtx, 16 Gb of ram, 500gb of ssd, running Ubuntu 18.10).

Thanks in advance for comments and or suggestions.

edit maybe I should also mention that I have ~42600 images in my library, and I have a loose organization of these images in only a few main folders. So for example I have 2 folders (“personal”, “travel”) with around ~16k and ~12k images in them, all stored on an external drive.

2

This is a known issue currently being fixed in master by @Pascal_Obry . Sit tight. :grinning:

1 Like
3

Well that’s good to hear! I knew it didn’t feel right, but I also thought “perhaps guys use darktable mainly for its processing capabilities and less for library maintenance etc, or have smaller libraries”…

BTW, bravo for your work on filmic! I’m still trying to get the best out of it and many times I resort to a more conventional tone curve out of frustration, but it’s also true I need to spend some more time on it and build some experience.

1 Like
4

What is the definition of the screen ?

For large screen 4k and more, the lighttable speed (new implementation in master) is day and night compare to the current one. I suppose the speed factor gain is more than 20x and this figure is not linear as on 5k screen the perf was even worst.

5

It’s a 27" Benq 2560x1440 but I see the same performance also when using only the internal display of my Dell which is a simple full hd display.

So the implementation of the new light table is already available? I only need to compile DT from source?

6

Yes, from master. But remember to backup your current config. You won’t be able to move back as the blend params have been updated.

7

Thanks, I’ll give it a go then! Can I ask how did you increase the performances? Was there a bug or did you change anything drastically?

8

No bug, I have changed the way the redraw was done. The rewrite is drastic indeed. For the record we were redrawing the whole lighttable for every pixel move of the mouse cursor. Now:

  • the mouse cursor events are filtered to keep only those that can have an effect
  • the redraw when needed is done only for the thumbs (including over highlight) that may have changed
  • the whole redraw is still needed when changing collection, changing filter, etc…
4 Likes
9

@Pascal_Obry ehm maybe it’s a stupid question but is the 2.7 package available here sufficiently up to date to include the modifcations to lighttable that you were discussing?:

https://software.opensuse.org/download.html?project=graphics:darktable:master&package=darktable

(reason being that it’s already compiled obviously)

10

The OBS repo should be very up-to-date, as it builds from got master.

11

@aadm: yes, those packages should have the lighttable speed-up. but as usual with dev packages, backup your darktable data and also not that you won’t be able to downgrade (or you’ll have to restore your backup).

12

Ok well in the end I cloned the git repository, installed the extra stuff I needed to compile darktable and the process was quick and painless. So I now have the latest build, see the screenshot below reporting the latest commit of yesterday:

Screenshot%20from%202019-01-13%2010-35-43

And what I’ve found from a first test is that indeed zipping around the lighttable is much faster than before, but then the the lag between using keyboard shortcuts to set stars and color labels feels the same as before. I have then tried to clicking the star icons on the bottom of each thumbnail and that is instantaneuous, no lag at all (not sure if that was also the case before as I normally use keyboard shortcuts to do this). On the other hand, clicking on the color label in the bottom of the screen:

Screenshot%20from%202019-01-13%2010-42-52

…that is also slow (~1sec from clicking to actually seeing the color label appearing on the thumbnail.

So there it is. For now I’ll sneak my way around using the mouse and only stars to do my editing (meaning selecting photos), but I think it could be helpful to make the editing process quicker by allowing the user to move around the lighttable and setting stars and labels and keywords (tags) with keyboard shortcuts and these actions should have a nearly-instantaneous feedback on screen.

I hope you can find these findings useful and reproduceable. Having gone through the binary building process I could help in testing new releases if you want.

Two other things I’ve noticed:

  1. in this build I don’t have access anymore to the lua script manager (it used to appear on the left panel and gave me access to quicktags, very useful!).
  2. there is a bug when scrolling the lighttable with the keyboard: you go down to the last row of thumbs and then you go down one more row and what gets updated is only the thumb under the selected thumb, not the entire grid. In other words, it looks like everything is fixed in place and you’re not scrolling down the grid – except for the single thumb currently under the cursor (I hope my explanation is clear).
13

For:

  1. you need to have the lua dev library for the lua support to be enabled
  2. I can reproduce and I’m sure this was working… a regression! I’ll fix.

For the slowness, this is because when a star or color-label is added we redraw everything as if the lighttable is using a corresponding filter we need to refresh it completely. I suppose that we can do that only if the corresponding filter is activated, for filename order there is no need for the whole redraw. I’ll try to improve that next…

1 Like
14

Your second point is fixed in master.

15

Applying rating and colorlabels from keyboard should be faster now. If you can test again that would be nice.

2 Likes
16

Noticeable speedup in the current master when using tags and colors/star ratings :smiley: :+1:

17

Ok so just tried the latest version, the bug we discussed (point 2 above) is fixed, great! About how fast are stars and labels applied from keyboard I’m sorry but I’m not seeing any difference from before… But @matejmarti did notice a speedup!

PS: sorry again for being such a newbie but I have installed lua, compiled 5.3 directly from source and then installed so I now have the bin and the libraries all on my system (under /usr/local but that’s in the path), however when compiling darktable I do get this message like it’s not finding lua:

-- Checking for one of the modules 'lua53;lua5.3;lua-5.3;lua'
-- Lua support: System library for lua 5.3 not found (to use darktable's version use -DDONT_USE_INTERNAL_LUA=Off)
18

I’ve used the buttons in the gui to apply the color labels during my short test last night. I will check again today using the keyboard shortcuts.

19

@aadm: for Lua sorry I’m no expert, you may try to remove your build directory and restart from scratch.

20

You know I’ve checked how’s the situation by using only the gui and as before, stars are ok, they are applied instantaneously, but color labels are not. I wonder what’s the difference with your setup. Do you have a more powerful computer? Is your library perhaps smaller (mine is close 50k images now and they are all contained in only a few directories; again I don’t think this should be an issue but you never know).

Anyway, the problem remains when using keyboard shortcuts for both stars and color labels.

I will also say that the same lag occurs when applying tags.