OpenCL problem DT 4.6 Windows

Terry · December 31, 2023, 1:07am

This has been a persistent problem on my computer and updating the Nvidia Drivers this week and installing DT4.6 has not resolved my problem.

I recently bought a Canon R7 and the CR3 files from that cause DT to crash when I move the details threshold slider in the Diffuse or Sharpen module. I previously raised the issue in this thread Open CL question darktable V4.5.0+816. My work around has been to allocate all device memory to darktable. This allowed me to keep using DT but is not a true fix. Today I opened a 52.3MB nef file from play raw My eyes! Too much bokeh!
and this file caused a crash even with all device memory allocated to darktable when I tried using the details threshold slider. I have attached a crash log from today with a CR3 file when I set default values for open CL and then tried using the details threshold slider. I hope this can be informative.

darktable-log.txt (458.7 KB)

g-man · December 31, 2023, 1:21am

delete your current logfile so we create a new one from now on
disable the amd GPU via the processing preference menú option
try again with the same xmp

Can you share the xmp?

g-man · December 31, 2023, 1:58am

Also:

Can you post how much available memory windows reports for your system (with your normal web browser open)? A print screen will be good.
use -d opencl -d memory instead of -d common

Terry · December 31, 2023, 2:52am

disabling amd did not help. Here is the new crash file and the xmp file. I am not sure how to get the system memory report that you request. My computer sadly is limited to 16GB RAM

darktable-log.txt (490.3 KB)
2R0A0494.CR3.xmp (9.7 KB)

g-man · December 31, 2023, 3:20am

First, Delete the logfile so we don’t get all of the old stuff.

It looks like AMD is still on. I think need to restart dt after that change.

Just use -d opencl

Terry · December 31, 2023, 5:53am

Thanks for your patience. I thought a new text file was written every time and for some reason my computer takes nearly 5 minutes to find that file by searching the C drive. The INETCache folder seems hidden from me despite asking windows to show hidden folders. I have now deleted the crash file, set DT up without AMD and opened it via command line. Here is the crash report. I hope it can be helpful.
darktable-log.txt (8.7 KB)

kofa · December 31, 2023, 7:46am

You need to disable ‘hide protected operating system folders’.

But you don’t have to search: you can type. Maybe even bookmark it, once you have it open.

g-man · December 31, 2023, 1:25pm

I was sure the iGPU from AMD was causing problems with using system memory. I was wrong.

Keep the AMD off.
Turn off the Nvidia (does it fault with just CPU path?)
Go back to -d common

Can you share the associated CR3 to the xmp you posted above?

I will try on my windows build later today/tomorrow.

priort · December 31, 2023, 3:41pm

I just loaded the xmp on to the original CR3 and put it on the NEF mentioned above. THere are two instances of D&S… One has a mask so I assumed this was the one initiating the problem… I could see no issue. I also bumped iterations all the way to 128 the max slider value and went repeatedly back and forth from +100 to - 100 on the details and I have no issues… My box is a 12 gen intel, running windows 11 with a 3060TI card and 32Gb system ram. So maybe this is an amd or resources issue??

EDIT:

These are my settings. I did not that long ago move it to use all device memory to see if there was any issue. I have not had any crashes so I have left it there… I did not look to see if I had any benefit from doing so… otherwise… everything disabled but Nvidia which is my GPU…

g-man · December 31, 2023, 6:18pm

I managed to reproduced it. It on the opencl path with limited memory resources. I used resources with: resource_small=128 4 64 200 . Using 300 and 400 works ok. Yes I forgot about using mini. I will switch to Fedora to see if I could reproduce there and start a GitHub Issue.

     8.6980 blend with form CL         [full]           diffuse                (   0/   0) 1560x1041 scale=0.2234 --> (   0/   0) 1560x1041 scale=0.2234 IOP_CS_RGB, BLEND_CS_RGB_SCENE, no form
     8.6980 refine_detail_mask on GPU  [full]           diffuse                (   0/   0) 1560x1041 scale=0.2234 --> (   0/   0) 1560x1041 scale=0.2234 
     8.6980 [opencl memory] device 0: 125108204 bytes (119.3 MB) in use
     8.6980 [opencl memory] device 0: 190180444 bytes (181.4 MB) in use
     8.7085 [opencl memory] device 0: 190180496 bytes (181.4 MB) in use
     8.7112 [opencl memory] device 0: 190180444 bytes (181.4 MB) in use
     8.7247 [opencl memory] device 0: 125108204 bytes (119.3 MB) in use
     8.7249 [opencl memory] device 0: 60035964 bytes (57.3 MB) in use
     8.7252 distort detail mask        [full]           diffuse                (3490/   0) 3491x4660 scale=1.0000
     8.7252 distort detail mask        [full]           demosaic               (   0/   0) 6983x4660 scale=1.0000 --> (   0/   0) 1560x1041 scale=0.2234 
     8.7252 resample_1c_plain                                                  (   0/   0) 6983x4660 scale=1.0000 --> (   0/   0) 1560x1041 scale=0.2234 bicubic
Magick: caught exception 0xC0000005 "Access violation"...

g-man · December 31, 2023, 6:39pm

Reproduced on current master (4.7.0~git97.fcbe7dd5) Fedora 39 KDE 12gb Nvidia GPU
resource_small=128 4 64 200
Changing from lanczos3 to bicubic still crashes

     6.3712 blend with form CL         [full]           diffuse                (   0/   0) 1657x1106 scale=0.2373 --> (   0/   0) 1657x1106 scale=0.2373 IOP_CS_RGB, BLEND_CS_RGB_SCENE, no form
     6.3713 refine_detail_mask on GPU  [full]           diffuse                (   0/   0) 1657x1106 scale=0.2373 --> (   0/   0) 1657x1106 scale=0.2373 
     6.3713 [opencl memory] device 0: 110662836 bytes (105.5 MB) in use
     6.3713 [opencl memory] device 0: 153777156 bytes (146.7 MB) in use
     6.3760 [opencl memory] device 0: 153777208 bytes (146.7 MB) in use
     6.3764 [opencl memory] device 0: 153777156 bytes (146.7 MB) in use
     6.3965 [opencl memory] device 0: 110662836 bytes (105.5 MB) in use
     6.3967 [opencl memory] device 0: 67548516 bytes (64.4 MB) in use
     6.3977 distort detail mask        [full]           diffuse                (4668/   0) 2313x4660 scale=1.0000
     6.3977 distort detail mask        [full]           demosaic               (   0/   0) 6981x4660 scale=1.0000 --> (   0/   0) 1657x1106 scale=0.2373 
     6.3977 resample_1c_plain                                                  (   0/   0) 6981x4660 scale=1.0000 --> (   0/   0) 1657x1106 scale=0.2373 lanczos3
Segmentation fault (core dumped)

g-man · December 31, 2023, 7:19pm

Please continue the conversation here:

github.com/darktable-org/darktable

[Bug] D&S with mask segfaults with low OpenCL memory

opened 07:07PM - 31 Dec 23 UTC

gi-man

### Describe the bug Originally reported at https://discuss.pixls.us/t/opencl-p…roblem-dt-4-6-windows/41240/1 To reproduce, I had to setup darktablerc to use: `resource_small=128 4 64 **200**` I reproduced on Windows 11 4.6.0 and Fedora KDE current master (4.7.0~git97.fcbe7dd5) ### Steps to reproduce Use attached xmp or manually apply a mask to a D&S module and adjust the detail threshold. CR3 and xmp here: https://discuss.pixls.us/t/open-cl-question-darktable-v4-5-0-816/39914/52?u=g-man ### Expected behavior _No response_ ### Logfile | Screenshot | Screencast ``` 6.3712 blend with form CL [full] diffuse ( 0/ 0) 1657x1106 scale=0.2373 --> ( 0/ 0) 1657x1106 scale=0.2373 IOP_CS_RGB, BLEND_CS_RGB_SCENE, no form 6.3713 refine_detail_mask on GPU [full] diffuse ( 0/ 0) 1657x1106 scale=0.2373 --> ( 0/ 0) 1657x1106 scale=0.2373 6.3713 [opencl memory] device 0: 110662836 bytes (105.5 MB) in use 6.3713 [opencl memory] device 0: 153777156 bytes (146.7 MB) in use 6.3760 [opencl memory] device 0: 153777208 bytes (146.7 MB) in use 6.3764 [opencl memory] device 0: 153777156 bytes (146.7 MB) in use 6.3965 [opencl memory] device 0: 110662836 bytes (105.5 MB) in use 6.3967 [opencl memory] device 0: 67548516 bytes (64.4 MB) in use 6.3977 distort detail mask [full] diffuse (4668/ 0) 2313x4660 scale=1.0000 6.3977 distort detail mask [full] demosaic ( 0/ 0) 6981x4660 scale=1.0000 --> ( 0/ 0) 1657x1106 scale=0.2373 6.3977 resample_1c_plain ( 0/ 0) 6981x4660 scale=1.0000 --> ( 0/ 0) 1657x1106 scale=0.2373 lanczos3 Segmentation fault (core dumped) ``` ### Commit _No response_ ### Where did you obtain darktable from? OBS ### darktable version 4.7.0~git97.fcbe7dd5 ### What OS are you using? Linux ### What is the version of your OS? Fedora 39 KDE X11 ### Describe your system? 12GB 3060 Nvidia GPU ### Are you using OpenCL GPU in darktable? Yes ### If yes, what is the GPU card and driver? Nvidia 3060 12GB 545.29.06 ### Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip No LUA. Changing from lanczos3 to bicubic The OP on Pixls.us has a 4gb nvidia card and uses `default` for resource level. This should yield ~2.3Gb opencl memory (4-0.6) * (700/1024)

Terry · December 31, 2023, 8:48pm

Thanks for your help @g-man . Do you need anything else from me at this stage? Thanks @kofa I have setup folder view as you described. BTW, I replied here because I didn’t want to add noise to the bug report on Github.

Terry · December 31, 2023, 8:51pm

@priort do you want to try my image by unchecking use all device memory. Use all device memory prevents crashes on my system.

priort · December 31, 2023, 8:58pm

Changed to

DT

128 iterations on the module and moving the details slider freely from -100 to + 100 with no crash…

Terry · December 31, 2023, 8:59pm

Changing the darktable CPU/memory resources to large has also prevented the crashing on my computer.

EDIT: The nef file still crashes even with CPU/memory set to large or unrestricted and OpenCL set to use all device memory. The crash occurs using the details threshold slider. If I disable OpenCL I get no crashing with this image.

priort · December 31, 2023, 9:14pm

I can use small as well with no issue…

Terry · December 31, 2023, 9:16pm

@priort it is probably unique to my computer.

priort · December 31, 2023, 9:17pm

Do you need the intel support on …not sure if that messes with anything but you dont’ have any intel GPU do you??

Terry · December 31, 2023, 9:20pm

The intel seems to make no difference. Probably no reason to have it on. I am unsure why DT selected it by default.