Slow export in Darktable 3

Software darktable
1

Does anybody else experience significant, huge slowdowns in exporting images in darktable 3?

My system (Fedora 31) wanted to upgrade Darktable, to darktable-3.0.0-2.fc31. I thought nothing of it, in fact, I haven’t even noticed anything special until I first started darktable a few days after that, and it asked me to “convert the databases” or something. I’ve allowed it and started working. Some small changes I’ve noticed, but the biggest issue is that when I export photos (from RAW to JPEG), it takes a lot longer then it used to. With the previous versions, a typical export would take maybe up to a second or two - I can’t say because it was never a big deal. But with 3, it takes so much longer - about 20-30 seconds per image.

Is that just my problem or is anybody else noticing it? Is it something to do with some setting, or is it simply a regression that I’d have to report somewhere?

Thanks in advance.

1 Like
2

Do you have openCL enabled?

3

Did you use any “denoise” function?
Use denoise in darktable always slow on export files.

4

I do, I think this is something I haven’t been doing much before. Should it really bring the processing times to from a few seconds to a few minutes? Seems drastic!

5

Not sure, it’s a default installation on Fedora (on an older Thinkpad, with 16GB of RAM, but an older CPU and integrated GPU). I’m persuing the “denoise” thread first, maybe that’s the issue, but I’ll also check if OpenCL is enabled to rule it out.

Did the default setting for OpenCL change between the two versions? Because I’ve just installed the app from the Fedora software repos and I haven’t done any (relevant) config changes that I know of - neither on the 2.6 nor on the new version.

My problem is that one export lasted seconds, then after the upgrade, it lasts minutes.

6

Fedora had OpenMP disabled for quite some time. Also the JPEG exporter isn’t the most performant code. I hope AVIF support will be added to browsers this year. It is a lot faster especially with tiling and a multi core CPU :slight_smile:

1 Like
7

I’m on windows but my export varies quite a bit depending on the image. Examples: a portrait in B/W just exported to jpg in 5 sec. A older landscape with high dynamic range in colour took 25 sec. I thought that was normal behaviour, maybe its not?

8

I am on Fedora 31, OpenMP works just fine. All 8 cores are utilized during exporfs.

9

I already encountered the problem, I uninstalled and reinstalled

10

I have had some slowdowns recently. First was at first resolved by upgrading my kernel (5.3.0) to 5.4.x. And performance increase was huge (about 200% and not also on export).
Second thing, I have tested to export a 40Mb size RAW (my RAW are only 20-24Mb). With that RAW, I have had slowdown and even crash on export. I have 8Go of RAM and I was on 4000 Mo in “host memory limit of tiling” in core options of preferences dialog. Upgrading it to 6000 and reactivate Swap (deactivated by the fact that kernel 5.3.0 have some dealing problems with Swap ; issue solved in 5.4).
Hope this help.

11

I tested on a i7 Skylake quadcore. Profiled denoise alone makes the export ~10x slower, like from 1,5 seconds to 15 seconds with 20 MP RAW files. Maybe with larger files, a way slower machine and complex filter modules you get several minutes. You can just try to disable some modules to check which is the culprit.

12

Retouch is another performance killer - it’s easy to get 10x slowdown

1 Like
13

Hello everyone again,

Thanks for all your replies. I’ve done some testing and it does seem to appear that, depending on which modules I pick, the export times can skyrocket.

I’m just not sure how come I haven’t seen it before - I’ve probably done most of my “expensive” exports on my desktop, and compared to the laptop above, the difference is not so drastic.

In any case, good to know.

14

FYI, on a i7 8 cores (Linux Debian) with OpenCL and export full size @ 90% JPEG quality, I never exceed 6 sec. for 18 Mb RAW’s. All with heavy processing including denoise profiled.

91,271738 [dev_process_export] pixel pipeline processing took 3,637 secs (13,328 CPU)
94,183784 [dev_process_export] pixel pipeline processing took 2,719 secs (2,077 CPU)
97,675493 [dev_process_export] pixel pipeline processing took 3,265 secs (2,383 CPU)
101,499898 [dev_process_export] pixel pipeline processing took 3,582 secs (12,243 CPU)
103,203967 [dev_process_export] pixel pipeline processing took 1,209 secs (1,438 CPU)
105,568787 [dev_process_export] pixel pipeline processing took 1,914 secs (2,399 CPU)
109,393810 [dev_process_export] pixel pipeline processing took 3,390 secs (2,995 CPU)
115,440627 [dev_process_export] pixel pipeline processing took 5,497 secs (18,764 CPU)
119,168106 [dev_process_export] pixel pipeline processing took 3,271 secs (2,476 CPU)
119,929405 [dev_process_export] pixel pipeline processing took 0,279 secs (0,359 CPU)
122,371953 [dev_process_export] pixel pipeline processing took 1,892 secs (10,665 CPU)
127,563738 [dev_process_export] pixel pipeline processing took 4,713 secs (25,985 CPU)
129,640783 [dev_process_export] pixel pipeline processing took 1,543 secs (2,141 CPU)
130,334004 [dev_process_export] pixel pipeline processing took 0,240 secs (0,367 CPU)
1 Like
15

Hello,

I am having Darktabke crawl so slow. I don’t know what’s going on, yesterday it was all fine, today I try to export few hundreds images and it takes ridiculously long time. What’s even more worrying is it appears Darktable stopped using my computer resources the way it used to. I mean, to use only 12% of processing power available? That is very weird. At times it just stops, then wakes up and continues.

It never was like that. Similar batches used to take an hour to export, not more, now for entire day I don’t know how to finish exporting the image batch I need.

16

@Danas_Anis you’re going to need to provide way more details than that. Can you find the darktable log and post it, along with OS and dt version details?

17

If you did not update Darktable or change some config in the software, then maybe, Darktable just shows the symptoms of another problem in your computer.
Have you found other softs that seems slower or other problems ?

18

In my case (Windows, DT 3.2.1) the export to jpg takes minutes, not seconds.

19

Hardware used? OpenCL enabled/disabled? Modules used (denoising, masking)? …
There is a large number of parameters influencing the time needed for processing/exporting images. Without having any information on this parameters it is impossible to give any qualified support.
There is a debug option (-d perf) for dt which can be used to analyze in detail where dt needs how much time (see manual for further information).

20

I am doing further investigation, I may be having a HDD fault, which results in Darktable to behave slow, other software is having slowdowns with files from that HDD as well.