Darktable core options for i7/16gb ram/etc

Software darktable
1

Hi there, I’ve almost completed my switch from Lightroom+Mac, and I can say now that I’m so happy with Darktable+Linux – my congratulations to the Darktable team that has really done an incredible job.

After getting a new laptop this summer (Dell XPS15 with 16Gb of Ram, Core i7-7700HQ @2.8-Ghz, Gefore GTX1050 and an ssd to store library data and thumbnails cache – the actual raws are on an external disk connected via USB3) I still feel DT is not as quick as Lightroom was on a less powerful computer (a macbook pro mid-2014, with core i5@2.6Ghz, 8Gb of ram etc).

I think I’ve fixed the browsing speed while in Lighttable mode by creating all the thumbnails separately with darktable-generate-cache, I’m now able to quickly preview images at full screen without any lag, but when in Darkroom mode even small m43 16Mp raw files do take a while to preview changes in exposure, tone curves etc. The problem is that the situation is even worse when working on files from my current camera (Fuji XT2, 24Mp files with the added complexity of the non-bayer sensor that I understand are slower to process).

I wonder if anyone has a recipe for the “core” options in the preferences and if they would have any effect in the darkroom processing; I did obviously read the relevant docs (e.g., https://www.darktable.org/2012/03/darktable-and-memory, https://www.darktable.org/usermanual/en/preferences_chapter.html) but not sure if they are still valid for my current computer. Anyway these are my current settings (reporting only those that I think have an effect on the actual speed):

core options

  • memory in megabytes to use for thumbnail cache: 2048
  • color managed cached thumbnails: UNCHECK
  • number of background threads: 4
  • host memory limit (in MB) for tiling: 1500
  • minimum amount of memor for a single buffer in tiling: 16
  • demosaicing for zoomed out darkroom: always bilinear
  • pixel interpolator: lanczos3

Since we’re here, can somebody also please explain to me the following GUI options:

GUI options

  • don’t use embedded preview JPEG but half-size raw: UNCHECK
  • high quality thumb processing from size: 720p

Would these two make a difference if I create thumbnails separately with darktable-generate-cache? As I can see if I leave the first one unchecked I still see the raws in lighttable with the same picture profile applied while shooting (for example b/w, the raw will then switch to color when opened in darkroom). But is this faster than using the actual thumbnails created by darktable or will it be the same when browsing the library? And the second option, is it related to the --max-mip option of darktable-generate-cache?

Note

I’m now using Darktable 2.6rc1 but the situation regarding speed hasn’t changed much from 2.4 that I was using before.

1 Like
2

Does OpenCL work on your machine?

darktable-cltest

3

yes OpenCL does work (haven’t tried this command line you wrote however).

4

You can also try running “darktable -d perf -d opencl”, then check the terminal output to see what exactly is taking so long after updating exposure etc.

5

In the settings have you selected “very fast GPU”?

6

This one means that you don’t want the lighttable to use the embedded (in the RAW) jpeg for preview. So dt will process the RAW (once for a specific thumb size) to create the preview in the lighttable.

Generally this is not an issue and I’m using the same option. I don’t like having the jpeg preview from the camera on the lighttable displayed first and when entering the darkrroom a different rendering.

7
  • memory in megabytes to use for thumbnail cache: 256
  • enable disk backend for thumbnail cache : CHECK
  • color managed cached thumbnails: CHECK
  • number of background threads: 8
  • host memory limit (in MB) for tiling: 7979
  • minimum amount of memor for a single buffer in tiling: 16
  • demosaicing for zoomed out darkroom: at most PPG
  • pixel interpolator: lanczos3

And for me darktable is very fast for whatever. Note that I have OpenCL activated too.

1 Like
8

The documentation also specifies that certain modules are more ‘processing intensive’ and should be activated at the end of the processing session. This can also cause some delays when clicking around in the interface.

9

Thanks @Pascal_Obry and all the others for the replies. I wonder why you’re keeping the thumbnail cache megabytes so low… I thought this would help by making it larger?.. anyway, I’ll try to reduce mine and see if that makes a difference. I will also increase the number of background threads and the host memory limit for tiling.

10

I have a gtx1070 in one of my machines and it is blazing fast with OpenCL. You need the nVidia proprietary driver for OpenCL to work.

11

I have the nvidia driver version 390.77 (as I can see from NVIDIA X Server settings).

As an aside, and this obv. has nothing to do with darktable, but I did try hard to make the thingie, whatsitcalled, bumblebee perhaps?, work so that I could only run darktable with the Geforce GTX and the rest with the onboard Intel graphics but I gave up… so now it’s constantly using the geforce and only when I have to unplug the notebook I will reboot in intel-graphics only mode.

12

yes Andreas, I also tried darktable-cltest and this is the output (only the first lines):

(base) aadm@aadm-XPS15:~$ darktable-cltest 
0.093971 [opencl_init] opencl related configuration options:
0.094019 [opencl_init] 
0.094027 [opencl_init] opencl: 1
0.094034 [opencl_init] opencl_library: ''
0.094042 [opencl_init] opencl_memory_requirement: 768
0.094050 [opencl_init] opencl_memory_headroom: 300
0.094057 [opencl_init] opencl_device_priority: '*/!0,*/*/*'
0.094064 [opencl_init] opencl_mandatory_timeout: 200
0.094079 [opencl_init] opencl_size_roundup: 16
0.094089 [opencl_init] opencl_async_pixelpipe: 0
0.094096 [opencl_init] opencl_synch_cache: 0
0.094106 [opencl_init] opencl_number_event_handles: 25
0.094115 [opencl_init] opencl_micro_nap: 1000
0.094124 [opencl_init] opencl_use_pinned_memory: 0
0.094133 [opencl_init] opencl_use_cpu_devices: 0
0.094143 [opencl_init] opencl_avoid_atomics: 0
0.094151 [opencl_init] 
0.094591 [opencl_init] could not find opencl runtime library 'libOpenCL'
0.094745 [opencl_init] could not find opencl runtime library 'libOpenCL.so'
0.095404 [opencl_init] found opencl runtime library 'libOpenCL.so.1'
0.095471 [opencl_init] opencl library 'libOpenCL.so.1' found on your system and loaded
0.186997 [opencl_init] found 1 platform
0.187035 [opencl_init] found 1 device
0.187438 [opencl_init] device 0 `GeForce GTX 1050' has sm_20 support.
0.187625 [opencl_init] device 0 `GeForce GTX 1050' supports image sizes of 16384 x 32768
0.187637 [opencl_init] device 0 `GeForce GTX 1050' allows GPU memory allocations of up to 1010MB
[opencl_init] device 0: GeForce GTX 1050 
     GLOBAL_MEM_SIZE:          4042MB
     MAX_WORK_GROUP_SIZE:      1024
     MAX_WORK_ITEM_DIMENSIONS: 3
     MAX_WORK_ITEM_SIZES:      [ 1024 1024 64 ]
     DRIVER_VERSION:           390.77
     DEVICE_VERSION:           OpenCL 1.2 CUDA
0.459313 [opencl_init] options for OpenCL compiler: -cl-fast-relaxed-math  -DNVIDIA_SM_20=1 -DNVIDIA=1 -I"/usr/share/darktable/kernels"
0.460260 [opencl_init] compiling program `demosaic_ppg.cl' ..
0.461175 [opencl_load_program] loaded cached binary program from file `/home/aadm/.cache/darktable/cached_kernels_for_GeForceGTX1050/demosaic_ppg.cl.bin'
0.461195 [opencl_load_program] successfully loaded program from `/usr/share/darktable/kernels/demosaic_ppg.cl'
0.483222 [opencl_build_program] successfully built program
0.483253 [opencl_build_program] BUILD STATUS: 0
0.483262 BUILD LOG:
13

Just to close up this thread, I am now satisfied with the performance of Darktable 2.6 in darkroom mode. I have changed my prefs according to @Pascal_Obry suggestions, use OpenCL everytime I launch Darktable (basically I switch to the intel integrated gpu only when I’m unplugged from the docking station and on the go and need to preserve battery).

I have now processed tens of Fuji raws and all the normal modules and other common operations (like zooming at 100% and then back out) are pretty fast. I only take care in activating the noise reduction modules at the end (I normally use two instances of profiled denoise as per manual’s recommendations and only for ISO>800 images).

…having said that, I do have another little problem related to speed, but I’ll open another thread since it seems to me it’s not really appropriate for this one (it’s about speed in lighttable mode now, specifically when tagging images).

2 Likes