VERY slow opening Fuji .RAF files in Darktable - use to be fast...

Software
#1

I have used Rawtherapee and Darktable on an infrequent basis for the past 8 months but mostly Rawtherapee. I was starting to switch over to Darktable to gain the database features and decided to go through some more youtube tutorials but noticed that now it takes considerable amount of time to open a .RAF file (20 seconds) vs .NEF file. Of course if I zoom into the image, I get the “working” message and another 20 second wait. I have no idea what has changed in Darktable or Windows 10 but Darktable is not usable due to the long delays. I tried uninstalling and reinstalling darktable but no difference. Demosaic is set to the default. Does anyone have any ideas what I can check to resolve this issue? Rawtherapee works just as before with no delays opening .RAF files. Thanks in advance!!
Greg

Darktable 4.4.2
Windows 10 22H2, up to date.
Ryzen 5 2600
24 GB ram
NVIDIA 1050 video card
Crucial SSD
Fuji XT2 camera

#2

Could you post a RAW file along with the XMP? Make sure it’s a file you have already worked on. Might be that you have accidentally activated a “heavy” module.

#3

Is your opencl on? Post a darktable -d perf

#4

_DSF0901.RAF.xmp (6.8 KB)
I tried uploading the .RAF but I keep getting errors at 44% of the upload…

#5

Yes, opencl is on. I need some help with your second request. “post a darktable -d perf”… Is this run darktable with argument “-d perf”? If so, I tried this and darktable started but where do I find the results?
Thanks

#6

You have to type this in windows (no folder clicks)

#8

Time to open a .RAF file is about the same on edited vs non-edited files.

#9

darktable-log.txt (464.8 KB)
d- perf produced no log output except start/stop times so I ran d-all.

#10

Note that in rawspeed, the RAF decoder is single threaded while the nef decoder is multi threaded, right @kmilos ?

#11

d- perf wont work. It should be -d perf

The -d all has 3k rows and I could not see one that opens the file in the darkroom.

I did notice some lua errors.

As a reference on my system an uncompressed RAF from raw.pixls.us:
[dev_process_image] pixel pipeline took 0.459 secs (1.046 CPU) processing Fujifilm - X100F - 14bit uncompressed (3_2).RAF

#12

My bad, “-d perf” does not give any log output.

#13

I was close in my 20 second estimate…
Any ideas what can be causing this???

45.3693 [dev_process_image] pixel pipeline took 23.904 secs (24.125 CPU) processing `_DSF0901.RAF’

version: darktable 4.4.2
start: 2023:08:28 18:48:47

19.6704 [dt_dev_load_raw] loading the image. took 0.220 secs (1.250 CPU)
21.2370 [histogram] took 0.000 secs (0.000 CPU) scope draw
21.3194 [dt_dev_process_image_job] loading image. took 0.000 secs (0.000 CPU)
21.3686 [histogram] took 0.000 secs (0.000 CPU) scope draw
21.4652 [dev_pixelpipe] took 0.000 secs (0.000 CPU) initing base buffer [full]
21.5330 [dev_pixelpipe] took 0.068 secs (0.062 CPU) [full] processed `rawprepare' on GPU, blended on GPU
21.5754 [dev_pixelpipe] took 0.042 secs (0.031 CPU) [full] processed `temperature' on GPU, blended on GPU
21.6026 [dev_pixelpipe] took 0.027 secs (0.031 CPU) [full] processed `highlights' on GPU, blended on GPU
21.9377 [histogram] took 0.000 secs (0.000 CPU) scope draw
21.9774 [histogram] took 0.000 secs (0.000 CPU) scope draw
29.3578 [histogram] took 0.000 secs (0.000 CPU) scope draw
33.8241 [histogram] took 0.000 secs (0.000 CPU) scope draw
33.8455 [histogram] took 0.000 secs (0.000 CPU) scope draw
33.8901 [histogram] took 0.000 secs (0.000 CPU) scope draw
45.2797 [resample_cl] plan 0.000 secs (0.000 CPU) resample 0.600 secs (0.594 CPU)
45.3210 [dev_pixelpipe] took 23.718 secs (23.953 CPU) [full] processed `demosaic' on GPU with tiling, blended on CPU
45.3310 [dev_pixelpipe] took 0.010 secs (0.016 CPU) [full] processed `exposure' on GPU, blended on GPU
45.3358 [dev_pixelpipe] took 0.005 secs (0.000 CPU) [full] processed `colorin' on GPU, blended on GPU
45.3410 [dt_ioppr_transform_image_colorspace_cl] IOP_CS_LAB-->IOP_CS_RGB took 0.004 secs (0.000 GPU) [filmicrgb]
45.3478 [dev_pixelpipe] took 0.012 secs (0.016 CPU) [full] processed `filmicrgb' on GPU, blended on GPU
45.3530 [dt_ioppr_transform_image_colorspace_cl] IOP_CS_RGB-->IOP_CS_LAB took 0.004 secs (0.000 GPU) [colorout]
45.3610 [dev_pixelpipe] took 0.013 secs (0.016 CPU) [full] processed `colorout' on GPU, blended on GPU
45.3682 [dev_pixelpipe] took 0.007 secs (0.000 CPU) [full] processed `gamma' on CPU, blended on CPU
45.3693 [dev_process_image] pixel pipeline took 23.904 secs (24.125 CPU) processing `_DSF0901.RAF'
45.4406 [histogram] took 0.000 secs (0.000 CPU) scope draw
45.5889 [dev_pixelpipe] took 0.000 secs (0.000 CPU) initing base buffer [preview]
45.5899 [dev_pixelpipe] took 0.001 secs (0.000 CPU) [preview] processed `rawprepare' on CPU, blended on CPU
45.5910 [dev_pixelpipe] took 0.001 secs (0.000 CPU) [preview] processed `temperature' on CPU, blended on CPU
45.5940 [dev_pixelpipe] took 0.003 secs (0.000 CPU) [preview] processed `highlights' on CPU, blended on CPU
45.6222 [dev_pixelpipe] took 0.028 secs (0.125 CPU) [preview] processed `demosaic' on CPU, blended on CPU
45.6254 [dev_pixelpipe] took 0.003 secs (0.000 CPU) [preview] processed `exposure' on CPU, blended on CPU
45.6302 [dev_pixelpipe] took 0.005 secs (0.109 CPU) [preview] processed `colorin' on CPU, blended on CPU
45.6318 [dt_ioppr_transform_image_colorspace] IOP_CS_LAB-->IOP_CS_RGB took 0.001 secs (0.000 CPU) [filmicrgb]
45.6949 [dev_pixelpipe] took 0.064 secs (0.516 CPU) [preview] processed `filmicrgb' on CPU, blended on CPU
45.6985 [dt_ioppr_transform_image_colorspace] IOP_CS_RGB-->IOP_CS_LAB took 0.003 secs (0.000 CPU) [colorout]
45.7042 [dev_pixelpipe] took 0.009 secs (0.016 CPU) [preview] processed `colorout' on CPU, blended on CPU
45.7074 [dev_pixelpipe] took 0.003 secs (0.000 CPU) [preview] processed `gamma' on CPU, blended on CPU
45.7162 [dt_ioppr_transform_image_colorspace_rgb] RGB-->RGB took 0.009 secs (0.000 CPU) [final histogram]
45.7998 [histogram] took 0.092 secs (0.109 CPU) final waveform
45.8008 [dev_process_preview] pixel pipeline processing took 0.321 secs (0.984 CPU)
45.8087 [histogram] took 0.003 secs (0.000 CPU) scope draw
46.0557 [histogram] took 0.003 secs (0.000 CPU) scope draw
46.0970 [histogram] took 0.003 secs (0.016 CPU) scope draw
56.6096 [histogram] took 0.003 secs (0.000 CPU) scope draw
60.7973 [histogram] took 0.003 secs (0.000 CPU) scope draw
#14

This. What are you using in this module? The image is separated into tile so it can fit into the GPU memory. More video memory can help.

1 Like
#15

I agree that a nvidia 1050 is no where near the latest gen but this video card worked fine a couple of months ago. Unless it is having issues… No issues with other software. (Rawtherapee, DaVinci Resolve, etc)

#16

What setting you have on that module with the raf file? I’m not at the PC, so I can’t see your setting/xmp right now.

Your card is 2gb, but we reserve 0.4 as headroom for other OS functions and then you selected default for memory, so it uses 70% of that. Change to large so it could use 90%. Maybe this makes larger tiles (faster processing).

There is another option using a new feature that I haven’t used. It is explained in the manual: j. advantage hint

https://darktable-org.github.io/dtdocs/en/special-topics/mem-performance/

Using this feature should direct dt to use the CPU instead GPU to avoid tiling. Again, I haven’t used this.

#17

Attached is a screen shot of the history and the modules that are on.
History
Modules ON

#18

I tried to force my system to not have enough memory (via a internal parameter). It creates the tiles but I’m not near 20s for that module using your module settings. My card is a 3060, so it has more processing units.

Did you try adjusting the resources to large.

#19

Yes I did. No change, still 23ish seconds

#20

Last thing to try. Go to the folder that has the log file. In there you will see some folders with the words kernel (maybe they include the word nvidia). Go ahead and delete them. When darktable starts, it will generate (compile) new ones using the nvidia drivers.

#21

I deleted the folders, restarted DT, opened the same file and no change in time to open. Sorry for the late reply. The message board wouldn’t let me post the message. Too many for a new user. I have to wait 15 hours. LOL