RustiCL is the preferred OpenCL driver instead ROCm on AMD systems.

Hi There-

Having given up on my Intel Arc B580 and switching to an AMD Radeon RX 7600, I just ran across this information about using RustiCL instead of the ROCm drivers. Needless to say, I had already gotten OpenCL running on ROCm before running across this info.

Should I switch over the RustiCL, and if so, how do I do it? Is there any great benefit in doing so?

Thanks for any advice you can give.

I run the Darktable AppImage on the Aurora Universal Blue Distro if that is of any importance.

Complete details below:
Operating System: Aurora 44
KDE Plasma Version: 6.7.2
KDE Frameworks Version: 6.28.0
Qt Version: 6.11.1
Kernel Version: 7.0.12-201.fc44.x86_64 (64-bit)
Graphics Platform: Wayland
Processors: 24 Ă— AMD Ryzen 9 9900X 12-Core Processor
Memory: 64 GiB of RAM (60.4 GiB usable)
Graphics Processor 1: AMD Radeon RX 7600
Graphics Processor 2: AMD Ryzen 9 9900X 12-Core Processor
Manufacturer: Gigabyte Technology Co., Ltd.
Product Name: B650 AORUS ELITE AX

1 Like

Do as you like :slight_smile:

  1. There have been a huge amount of issues related to rocm, as you will likely know, support for hardware varies a lot, rocm is depending on support by the distribution and that made problems too. Notorious are arch and derivates.
  2. Currently there is no measurable performance gain for rocm on most cards
  3. The darktable rusticl support is ways better, it’s also regularly updated by distributions, also we have good relations with the main rustcl dev.
  4. If you run into an OpenCL issue, chances for getting it fixed within a few days are pretty good for rusticl
6 Likes

Took a quick pass at getting it installed, bit of a problem on my distro so I’ll cycle back to it or maybe try in a distrobox. Thanks

1 Like

Have had no success getting RustiCL up and running on the AppImage version.
Can’t get it to work in a container (yet).
I can use it on the Flatpak, but as of now, the Flatpak has no AI features.
Fedora 44 still has not released a 5.6 version as of this date.

Looks like I’m back to Rocm until the situation changes.

Cheers!

Comparing processing with ROCm vs. RustiCL I’m getting 3s and 6.8s correspondingly. Just my standard darktable-cli test, not tried GUI work much with RustiCL yet.

Are there any tweaks for RustiCL performance? darktablerc?

Comparing darktable-cli -d opencl outputs the difference is mainly in denoiseprofile and diffuse:

RustiCL:

     3.3196 [dev_pixelpipe] took 1.452 secs (0.228 CPU) [export] processed `denoiseprofile' on GPU with tiling, blended on CPU
     7.3419 [dev_pixelpipe] took 2.169 secs (0.228 CPU) [export] processed `diffuse' on GPU with tiling, blended on CPU
... ... ...
     8.4770 [opencl_profiling] profiling device 0 ('rusticl AMD Radeon RX 7600'):
     8.4771 [opencl_profiling] spent  0.0087 seconds in denoiseprofile_precondition_Y0U0V0
     8.4771 [opencl_profiling] spent  0.4191 seconds in denoiseprofile_decompose
     8.4771 [opencl_profiling] spent  0.0191 seconds in denoiseprofile_reduce_first
     8.4771 [opencl_profiling] spent  0.0002 seconds in denoiseprofile_reduce_second
     8.4771 [opencl_profiling] spent  0.0716 seconds in denoiseprofile_synthesize
     8.4771 [opencl_profiling] spent  0.0265 seconds in [Copy Image (on device)]
     8.4771 [opencl_profiling] spent  0.0069 seconds in denoiseprofile_backtransform_Y0U0V0
     8.4772 [opencl_profiling] spent  0.7973 seconds in diffuse_pde
... ... ...
     8.4772 [dev_process_export] pixel pipeline processing took 6.839 secs (4.062 CPU)

ROCm:

     2.3038 [dev_pixelpipe] took 0.529 secs (0.007 CPU) [export] processed `denoiseprofile' on GPU with tiling, blended on CPU
     4.1827 [dev_pixelpipe] took 0.784 secs (0.012 CPU) [export] processed `diffuse' on GPU with tiling, blended on CPU
... ... ...
     4.6439 [opencl_profiling] profiling device 0 ('AMD Accelerated Parallel Processing gfx1102'):
     4.6440 [opencl_profiling] spent  0.0065 seconds in denoiseprofile_precondition_Y0U0V0
     4.6440 [opencl_profiling] spent  0.0847 seconds in denoiseprofile_decompose
     4.6440 [opencl_profiling] spent  0.0219 seconds in denoiseprofile_reduce_first
     4.6440 [opencl_profiling] spent  0.0001 seconds in denoiseprofile_reduce_second
     4.6440 [opencl_profiling] spent  0.0698 seconds in denoiseprofile_synthesize
     4.6440 [opencl_profiling] spent  0.0268 seconds in [Copy Image (on device)]
     4.6440 [opencl_profiling] spent  0.0065 seconds in denoiseprofile_backtransform_Y0U0V0
     4.6440 [opencl_profiling] spent  0.1547 seconds in diffuse_pde
... ... ...
     4.6441 [dev_process_export] pixel pipeline processing took 3.035 secs (3.335 CPU)

But also, MAX MEM ALLOC difference?:
RustiCL:

   GLOBAL MEM SIZE:          8192 MB
   MAX MEM ALLOC:            2048 MB
   MAX CONSTANT BUFFER:      65536 KB
   ADDRESS ALIGN:            512
   COMPUTE UNITS:            32
   MAX WORK GROUP SIZE:      1024

ROCm:

   GLOBAL MEM SIZE:          8176 MB
   MAX MEM ALLOC:            6950 MB
   MAX CONSTANT BUFFER:      7116390 KB
   ADDRESS ALIGN:            256
   COMPUTE UNITS:            16
   MAX WORK GROUP SIZE:      256


Mostly decompose and diffuse_pde - will have a look.

About memory - that’s what the drivers report/provide. There is one restriction in rusticl (thats related to mesa) that might be of relevance for very large images, MAX MEM ALLOC of 2GB with 16b/pixel means that we can’t process images with more than 125Mpixels on GPU.

Test image is 50Mpx (OM System high resolution mode).

The difference is most visible for those two modules, but it exists for all. So perhaps it’s just the processing capabilities of RustiCL, not an issue with dt kernels? Full outputs now attached, in case that might be helpful.

gpu_radeonsi.txt (11.8 KB)
gpu_rocm.txt (11.9 KB)

Here is the side-by-side comparison of clpeak runs on the same system:

clpeak.pdf (40.3 KB)

You know you are using dt 5.4 ?

Unfortunately the logs don’t have a date logged, how old are they? All i can say, results differ - if at all - only by small margins, in some cases current rusticl is even faster.

So - as said - take as you like :slight_smile:

Logs are from yesterday, July 20. Versions are the latest from Debian “Sid of the day”.

This was only for the sake of testing ROCm vs RustiCL (I’m happy with Intel Arc). Will retest with 5.6.1 when it shows up on Sid.

Ok. Glad to hear you are fine with the intel, from time to time there seem to be problems with their driver, sometimes dropping support for some hardware so it seems.
Unfortunately i never got good logs so i couldn’t track that down and mostly i suspected not-a-driver-issue but user-installing issues.

About the AMD driver status, both systems improve performance with a steady pace. Hoping for atomic floats as we have on nvidia for long :slight_smile: (karol is on the track).

Following up on the memory discussion above, from a device that makes unified memory very visible: an AMD BC-250 (Cyan Skillfish, gfx1013) with 16 GB of unified GDDR6 shared between CPU and GPU.

First, a RustiCL packaging gotcha. On Fedora 44, darktable couldn’t compile basic.cl at all — Internal compilation error: nir_shader not fully linked, so the device was rejected and the OpenCL option stayed greyed out. The cause is that Fedora builds Mesa against upstream libclc rather than the patched mesa-libclc fork RustiCL expects. RustiCL warns about it on stderr, but it’s easy to miss. Building mesa-libclc and rebuilding Mesa against it fixed it entirely. Filed downstream — but worth knowing, since “RustiCL doesn’t work” reports on Fedora may share this cause.

Second, on memory. hannoschwalm noted that RustiCL’s 2 GB MAX MEM ALLOC limits GPU processing to ~125 Mpixel images. On a unified-memory device something else binds much earlier:

GLOBAL MEM SIZE:    16896 MB
MAX MEM ALLOC:      2048 MB
UNIFIED MEM SIZE:   3799 MB reserved

That 3799 MB is about a quarter of system RAM, and it did not change under resourcelevel=large, resourcelevel=unrestricted, opencl_tune_headroom=TRUE, or raising the kernel GTT limit so the device advertised 16.5 GiB instead of 8 GiB.

Exporting a 48 Mpixel raw, denoiseprofile, lens and diffuse all tile (and so blend on CPU), while bilat and hazeremoval fall back to CPU entirely — bilat runs on GPU happily at smaller sizes, so that’s memory pressure rather than a missing kernel. Net result is 1.75× over CPU.

A quarter of system RAM seems a sensible default for an APU sharing DDR, and quite conservative when the GPU has 16 GB of GDDR6 behind it. Is that reservation intentional for unified-memory devices, and would making it configurable be reasonable?

(Minor aside for the RustiCL side: in the tiled export, Read Image (device→host) was the largest single entry in the command queue at 2.0 s of 5.8 s. On a device where CPU and GPU share physical memory, that copy shouldn’t need to happen.)

Just have a look at AMD Rusticl OpenCL won't compile with llvm22 (upstream issue) · Issue #21662 · darktable-org/darktable · GitHub and the discussion with main rusticl developer , you understand why rusticl is recommended for dt?

  1. A friendly reminder we have the docs! You’ll find exactly what you are asking for to tune the shared unified memory size.
  2. More OpenCL related stuff has already be done for 5.8
  3. Some _ Read Image (device→host)_ are simply required for internal algorithms …
1 Like

Apologies for missing the unified memory size settings in the docs - setting the OpenCL memory fraction to 0.5 took the reserved size from 3799 MB to 7598 MB. Also, thanks for the pointer to #21662. I hit the llvm22 issue without realizing it was already tracked…

1 Like