Slow export in Darktable 3

OpenCL - not available
Modules used: exposure, filimic, local contrast, denoise (profiled), sometimes tone equalizer.
It is slow independently of modules used, but the biggest impact seems to have denosie and tone equalizer, which does not come as a surprise.
It is probably related to my old HW (i5 @ 2,7GHz, 4G RAM).

Still, all this waiting to get a JPG is not discouraging me from using DT :wink:

@AdamD you need at least another 4gb of ram.

I use darktable with 4 GB, but with an NVidia card and OpenCL support. Without tone equalizer and dehaze in ‘local average’ mode, performance is not an issue. denoise profiled, filmic, LUT 3D etc. A 16 MPixel image is exported in about 10 seconds, 2.5-4 of those are taken by the ‘Amaze’ demosaic.

More memory is better, but simply adding 4 GB more if the machine is not using much swap AND not using tiling too much won’t make much difference, I think. I’ve increased the tiling limit from 500 to 1000 MB, now I don’t get tiling in ‘Amaze’, and it still takes 2.6 seconds :slight_smile:

@AdamD: If you can live with lower-resolution exports, you may try entering an export size limit (such as 1920x1080 for FullHD, or whatever you prefer/need) in the export module and disable ‘high quality resampling’. Try exporting a few images with high quality en- and disabled. If you see no difference, I think you’ll be happy about the performance boost.

Which CPU is that, where Amaze needs 2.6 seconds for a 16MP File?
On my 8 years old AMD FX8350 CPU Amaze needs 0.19 seconds for a 15MP file (measured in RawTherapee)

Core2 Duo E6550, 2.33GHz :smiley:
https://www.cpubenchmark.net/compare/Intel-Core2-Duo-E6550-vs-AMD-FX-8350-Eight-Core/941vs1780

Anyone using dt on a laptop with Intel uhd graphics, or a new ryzen 4000 apu? Does opencl work ok on these?
Specifically in thinking of Acer Swift 3 with ryzen 4500U.

The intel/beignet implementation of OpenCL is blacklisted for darktable (darktable/opencl_drivers_blacklist.h at master · darktable-org/darktable · GitHub). So I assume OpenCL will be diabled for intel (uhd) graphics cards. For Ryzen 4500U I don’t know, but this APU contains only 6 GPU cores. Even if OpenCL is supported I would not expect a great performance boost (this is an assumption without having experiences with this type of processors).

Aww. So a thin and light ultrabook is not a good option for dt. Will have to start browsing through gaming laptops.

You can also look at eGPUs.

Yes, a eGPU migth be a solution to power up a low performance laptop, particular considering that you can run into temperature problems if you highly stress a (not optimally cooled) laptop with computational intensive tasks.

AdamD, I don’t know if you can install opencl on windows, but I know on Linux the difference with/without is quite significant (basically, for me I can use the external GPU or not)

Intel integrated UHD graphics: Yes, openCL works. At least it does for me. Asus Zenbook flip UX360UA, Core i5, Skylake GT2 [HD Graphics 520], linux mint 20, darktable 3.2.1.

Thin and light laptops are an option, may run hot at times (the above gets up to around 55-60K when exporting). Before you purchase, check on the quality of the screeen in terms of % coverage of sRGB/ Adobe RGB and screen brightness. A while ago I was thinking of upgrading to a Zephyrus G14 but decided aginst it as the screen brightness wasn’t great.

EDIT I just looked at a different review of the G14 and the screen came ot pretty good…time for another look.

Would be great having a laptop working at 55-60 Kelvin…, but I greatly fear that you meant 55-60 °C :wink: .

1 Like

The G14 and G15 have good screens, and are quite thin and light. It’s the cheaper A15 that has a poor quality screen.

Oh yes. Oops no liquid helium cooling here. It does get higher tho, I had it up to 92 centigrade streaming a 4k video (in 4k) from youtube on a hot day. Measured by output from “sensors”.

Just to clarify the difference between this:

and this:

Around the 6000 series Intel switched from the old Beignet (broken) driver to the NEO driver. The latter should be available for any new Intel CPU and works under Linux (on Windows is still blacklisted).

As others have mentioned, the integrated graphics can give you some boost but it’s no match for a dedicated GPU. It all depends of what you do, without OpenCL darktable works fine for light work, but if you plan to do heavy edits and many exports per session, a GPU is recommended.

Sorry for asking a question which has been answered numerous times - how to check export times in windows? I’d like to run the tests with and without opencl. And I’ll get a few friends to download dt and test a few different laptops.

Edit: Found the log file

13.656150 [dev] took 0.000 secs (0.000 CPU) to load the image.
13.886015 [dev_pixelpipe] took 0.018 secs (0.047 CPU) initing base buffer [full]
13.921377 [dev_pixelpipe] took 0.034 secs (0.016 CPU) processed `raw black/white point' on GPU, blended on GPU [full]
13.936195 [dev_pixelpipe] took 0.014 secs (0.000 CPU) processed `white balance' on GPU, blended on GPU [full]
14.054135 [dev_pixelpipe] took 0.117 secs (0.438 CPU) processed `highlight reconstruction' on CPU, blended on CPU [full]
14.150938 [dev_pixelpipe] took 0.096 secs (0.016 CPU) processed `demosaic' on GPU, blended on GPU [full]
14.894276 [dev_pixelpipe] took 0.741 secs (0.016 CPU) processed `denoise (profiled)' on GPU, blended on GPU [full]
14.920256 [dev_pixelpipe] took 0.024 secs (0.016 CPU) processed `lens correction' on GPU, blended on GPU [full]
14.932069 [dev_pixelpipe] took 0.009 secs (0.000 CPU) processed `crop and rotate' on GPU, blended on GPU [full]
14.943195 [dev_pixelpipe] took 0.008 secs (0.000 CPU) processed `exposure' on GPU, blended on GPU [full]
15.672742 [dev_pixelpipe] took 0.729 secs (2.609 CPU) processed `tone equalizer' on CPU, blended on CPU [full]
15.727480 [dev_pixelpipe] took 0.054 secs (0.016 CPU) processed `input color profile' on GPU, blended on GPU [full]
15.739776 [dev_pixelpipe] took 0.012 secs (0.000 CPU) processed `color balance' on GPU, blended on GPU [full]
image colorspace transform Lab-->RGB took 0.013 secs (0.062 CPU) [filmicrgb ]
22.584542 [dev_pixelpipe] took 6.845 secs (25.312 CPU) processed `filmic rgb' on CPU, blended on CPU [full]
image colorspace transform RGB-->Lab took 0.010 secs (0.000 GPU) [bilat ]
22.731932 [dev_pixelpipe] took 0.146 secs (0.047 CPU) processed `local contrast' on GPU, blended on GPU [full]
23.045852 [dev_pixelpipe] took 0.313 secs (0.047 CPU) processed `local contrast 1' on GPU, blended on GPU [full]
23.140676 [dev_pixelpipe] took 0.095 secs (0.016 CPU) processed `local contrast 2' on GPU, blended on GPU [full]
23.158605 [dev_pixelpipe] took 0.017 secs (0.000 CPU) processed `output color profile' on GPU, blended on GPU [full]
23.187825 [dev_pixelpipe] took 0.026 secs (0.000 CPU) processed `framing' on GPU, blended on GPU [full]
23.255588 [dev_pixelpipe] took 0.067 secs (0.156 CPU) processed `display encoding' on CPU, blended on CPU [full]
23.261584 [dev_process_image] pixel pipeline processing took 9.395 secs (28.766 CPU)
35.515712 [dev] took 0.000 secs (0.000 CPU) to load the image.
35.740573 [export] creating pixelpipe took 0.205 secs (0.344 CPU)
35.757564 [dev_pixelpipe] took 0.017 secs (0.047 CPU) initing base buffer [export]
35.785544 [dev_pixelpipe] took 0.028 secs (0.047 CPU) processed `raw black/white point' on GPU, blended on GPU [export]
35.800535 [dev_pixelpipe] took 0.014 secs (0.000 CPU) processed `white balance' on GPU, blended on GPU [export]
35.953441 [dev_pixelpipe] took 0.152 secs (0.484 CPU) processed `highlight reconstruction' on CPU, blended on CPU [export]
36.017933 [dev_pixelpipe] took 0.063 secs (0.000 CPU) processed `demosaic' on GPU, blended on GPU [export]
38.347763 [dev_pixelpipe] took 2.329 secs (0.422 CPU) processed `denoise (profiled)' on GPU, blended on GPU [export]
38.373086 [dev_pixelpipe] took 0.024 secs (0.000 CPU) processed `lens correction' on GPU, blended on GPU [export]
38.403142 [dev_pixelpipe] took 0.029 secs (0.000 CPU) processed `crop and rotate' on GPU, blended on GPU [export]
38.434790 [dev_pixelpipe] took 0.031 secs (0.000 CPU) processed `exposure' on GPU, blended on GPU [export]
41.497461 [dev_pixelpipe] took 3.062 secs (10.906 CPU) processed `tone equalizer' on CPU, blended on CPU [export]
41.597273 [dev_pixelpipe] took 0.099 secs (0.031 CPU) processed `input color profile' on GPU, blended on GPU [export]
41.625778 [dev_pixelpipe] took 0.028 secs (0.000 CPU) processed `color balance' on GPU, blended on GPU [export]
image colorspace transform Lab-->RGB took 0.042 secs (0.141 CPU) [filmicrgb ]
60.761935 [dev_pixelpipe] took 19.135 secs (70.688 CPU) processed `filmic rgb' on CPU, blended on CPU [export]
image colorspace transform RGB-->Lab took 0.020 secs (0.000 GPU) [bilat ]
61.095145 [dev_pixelpipe] took 0.332 secs (0.016 CPU) processed `local contrast' on GPU, blended on GPU [export]
61.718324 [dev_pixelpipe] took 0.623 secs (0.031 CPU) processed `local contrast 1' on GPU, blended on GPU [export]
61.901420 [dev_pixelpipe] took 0.182 secs (0.016 CPU) processed `local contrast 2' on GPU, blended on GPU [export]
61.931625 [dev_pixelpipe] took 0.029 secs (0.000 CPU) processed `output color profile' on GPU, blended on GPU [export]
61.962589 [dev_pixelpipe] took 0.030 secs (0.000 CPU) processed `framing' on GPU, blended on GPU [export]
62.101505 [dev_pixelpipe] took 0.139 secs (0.328 CPU) processed `display encoding' on CPU, blended on CPU [export]
62.102505 [dev_process_export] pixel pipeline processing took 26.362 secs (83.031 CPU)
[export_job] exported to `D:\Photos\DT Play Raw\JPG\_MG_7830_06.jpg'

Does this look ok?

2nd Edit: Tested on friend’s laptops.

LG Gram with i5 1035G7 and Intel UHD - 34sec. Does not use opencl.

HP Omen with 4800H and rtx2060 - 9sec.

3rd Edit: Sorry for the edits. I figured it’ll be better to keep this in a single post.

desktop i7 6700k + gtx1060 - 10.8secs.

I’ve always found that the be the case on my Windows machine. The first couple of exports go well and then the process bogs down to a crawl, typically about 5 to 10 minutes per file. A clean reboot gets DT back on its feed. It has the appearance of a memory leak. In addition, OpenCL locks up my machine altogether

Granted it may very well be a hardware issue on my computer. I’ll be glad to provide details if someone could be kind enough to let me know what I need to provide.

Update. Wrt to the results posted above, the 26secs I got on my desktop were with the raw file being read from an hdd. When I put it on ssd and tested, I got a result of 18secs!

Conclusion - I have ordered a 1tb ssd.

If you surround your log by backticks ``` it will look even better :wink:

Just insert three backticks ``` before and after your log file