HDR DNG Export to jpg problem

kofa · January 29, 2024, 6:03pm

The reason the developers need the logs is that this part of the error message is very generic, and is therefore not sufficient to diagnose the specific issue.

Edit: your log file does not contain the so-called ‘backtrace’, so is basically useless, unfortunately.

JankoK · January 29, 2024, 6:08pm

Thank you for this additional information! I’ve been using the $(VERSION) variable in my export file naming scheme, but $(VERSION.NAME) is new to me, and I somehow failed to realize to use these variables in overlays and possibly elsewhere …

JankoK · January 29, 2024, 6:18pm

Sorry, as I’m not programmer, so my knowledge in this area is limited. After the crash I just renamed the old log file (as a backup, and to isolate the problem) and when provoking the crash again, this is what was created as a new log file.
Is there a way to create a more extended version of a log file?
And here the log file from before - it is fairly big, but I hope it can provide additional insight …
darktable-log.txt.7z (21.3 KB)

g-man · January 29, 2024, 6:30pm

We shared the link to the FAQ in this thread. We highlight that we need a -d common log. Now I’m going to copy/paste the content from the FAQ to make it super clear how to do this.

I was working with darktable and it suddenly just crashed! What should I do?*
(faq | darktable)
Don’t panic, sometimes it happens. If you can reproduce the crash, please file a [bug report], and send the so called “backtrace” file as well. You can find the location of this backtrace file in the folder where the crash dialog indicates. Generating a log of the crash can also aid in discovering the cause. The simplest way is to start Windows Command Prompt (cmd), navigate to C:\Program files\darktable\bin and start darktable via darktable.exe -d common or darktable -d opencl or darktable -d perf or to see all the options darktable -h. The log file will be generated in the hidden path listed above.

g-man · January 29, 2024, 6:35pm

What version of darktable? 4.6?

JankoK · January 29, 2024, 7:39pm

Thank you for your information!
Attached the log file obtained after start from command line (“C:\Program Files\darktable\bin>darktable.exe -d common”), and subsequent crash …
darktable-log.txt (140.2 KB)
Hope that helps …

kofa · January 29, 2024, 7:51pm

There was this bug, which only occurred for specific resolutions:

github.com/darktable-org/darktable

Crash when exporting (Windows)

opened 09:12AM - 14 Dec 23 UTC

closed 07:07AM - 18 Dec 23 UTC

c-h-r-i-s-t-i-a-n

priority: high bug: pending

### Describe the bug darktable crashes when exporting an image with settings ma…xheight 1080, use-hq-reampling yes. ### Steps to reproduce 1. Lighttable export tab 2. Set maxheigt 1080 ( 0 x 1080 ) 3. Set high-quality-resampling = yes 4. Hit "export" ### Expected behavior no crash should occur ### Logfile | Screenshot | Screencast _No response_ ### Commit _No response_ ### Where did you obtain darktable from? self compiled ### darktable version 4.5.0+1435~g6d4cfccf32-dirty ### What OS are you using? Windows ### What is the version of your OS? Windows 10 64 bit ### Describe your system? The installer was build using MSYS. ### Are you using OpenCL GPU in darktable? None ### If yes, what is the GPU card and driver? _No response_ ### Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip _No response_

Does it occur at full-resolution JPG export? Does it not happen at lower resolution TIF?

It should have been fixed in 4.6.0, I think, but maybe your problem is related?

g-man · January 29, 2024, 7:56pm

The problem was fixed on master. You can wait for 4.6.1 or use Bill’s build or the current nightly.

    71,0148 clip_and_zoom_roi_cl       [export]         demosaic               (   0/   0) 4942x3224 scale=1,0000 --> (   0/   0) 4941x3225 scale=1,0000 
    71,0148 [opencl copy_image] could not copy image on device 0: CL_INVALID_VALUE

The ROI output (3225) is larger than the input (3224), thus creating a failure.

g-man · January 29, 2024, 8:13pm

Do you have a separate GPU card? or this is an embedded iGPU? This is not related to your issue, but is a general problem we likely have with Windows and AMD. It seems like AMD is reporting the card as a dedicated and 20gb of memory, which I think it is incorrect.

[dt_opencl_device_init]
   DEVICE:                   0: 'gfx1100'
   PLATFORM, VENDOR & ID:    AMD Accelerated Parallel Processing, Advanced Micro Devices, Inc., ID=4098
   CANONICAL NAME:           amdacceleratedparallelprocessinggfx1100
   DRIVER VERSION:           3608.0 (PAL,LC)
   DEVICE VERSION:           OpenCL 2.0 AMD-APP (3608.0)
   DEVICE_TYPE:              GPU, dedicated mem
   GLOBAL MEM SIZE:          20464 MB
   MAX MEM ALLOC:            17177 MB
   MAX IMAGE SIZE:           16384 x 16384

I think this is the second time I’ve seen this.

JankoK · January 29, 2024, 8:31pm

This is a separate GPU card on a desktop PC.
Here the card details:

JankoK · January 29, 2024, 8:55pm

Excellent find!
Seing your post, I did some more testing with various output sizes. It seems that the crash occurs at settings where one (or maybe both) pixel dimensions are some “standard” values (i. e. 1080, 1920, 2560, …)
As soon as I export to some arbitrary format (e. g. 1500 × 1100) everything works fine. This is true for jpg, as well as for tiff files …

JankoK · January 29, 2024, 8:56pm

This is excellent news, thank you for the info!

JankoK · January 29, 2024, 9:00pm

Thank you for the information!

I would appreciate feedback information, if, and when it is available.

kofa · January 29, 2024, 9:22pm

There was this other bug, which was linked to the other as a duplicate, and closed. There the report said the crash occurred for some specific ratios:

github.com/darktable-org/darktable

DT 4.6.0 crashes on export with specific scaling ratios

opened 05:09PM - 24 Dec 23 UTC

closed 02:47PM - 26 Dec 23 UTC

Mark-64

scope: image processing Fixed in master

### Describe the bug DT crashes on export when specific scaling ratios are sele…cted. For example, on my Sony A6400 (6000x4000 native resolution), it crashes when the scaling ratio is an integer, for example 6000x4000, 3000x2000, 1200x800, 600x400 and so on. It does not crash when the scaling ratio is even slightly different, like 3000x2001. Also it does not crash with 2000x3000, irrespective of the image orientation. ![immagine](https://github.com/darktable-org/darktable/assets/43290988/7c810ac9-3384-4f62-9a69-5db62a95121e) ### Steps to reproduce 1) Export an image to file by setting an integer scaling ratio 2) See the crash ### Expected behavior not crash ### Logfile | Screenshot | Screencast See the last part of the log ``` 18,6828 pixelpipe starting CL [export] ( 0/ 0) 2995x2000 scale=0,4970 --> ( 0/ 0) 2995x2000 scale=0,4970 device=0 (nvidiacudanvidiageforcegtx10603gb) 18,6828 [dt_opencl_check_tuning] use 2671MB (headroom=ON, pinning=OFF) on device `NVIDIA CUDA NVIDIA GeForce GTX 1060 3GB' id=0 18,6829 modify roi IN [export] finalscale ( 0/ 0) 6026x4025 scale=1,0000 --> ( 0/ 0) 2995x2000 scale=0,4970 18,6829 modify roi IN [export] demosaic ( 0/ 0) 6026x4024 scale=1,0000 --> ( 0/ 0) 6026x4025 scale=1,0000 18,6829 modify roi IN [export] rawprepare ( 0/ 0) 6048x4024 scale=1,0000 --> ( 0/ 0) 6026x4024 scale=1,0000 18,6829 pixelpipe data: full [export] ( 0/ 0) 6048x4024 scale=1,0000 --> ( 0/ 0) 6048x4024 scale=1,0000 18,6829 [dev_pixelpipe] took 0,000 secs (0,000 CPU) initing base buffer [export] 18,6830 [opencl memory] device 0: 48674304 bytes (46,4 MB) in use 18,6897 [opencl memory] device 0: 145668800 bytes (138,9 MB) in use 18,6916 pixelpipe process CL [export] rawprepare ( 0/ 0) 6048x4024 scale=1,0000 --> ( 0/ 0) 6026x4024 scale=1,0000 IOP_CS_RAW 18,6916 [opencl memory] device 0: 145668816 bytes (138,9 MB) in use 18,6916 [opencl memory] device 0: 145668832 bytes (138,9 MB) in use 18,6962 [opencl memory] device 0: 145668816 bytes (138,9 MB) in use 18,6978 [opencl memory] device 0: 145668800 bytes (138,9 MB) in use 18,6981 [opencl memory] device 0: 96994496 bytes (92,5 MB) in use 18,6982 [dev_pixelpipe] took 0,015 secs (0,000 CPU) [export] processed `rawprepare' on GPU, blended on GPU 18,8087 [dev_pixelpipe] module `rawprepare' min: (0,000202) max: (0,374865) [export] 18,8088 [opencl memory] device 0: 193988992 bytes (185,0 MB) in use 18,8106 pixelpipe process CL [export] temperature ( 0/ 0) 6026x4024 scale=1,0000 --> ( 0/ 0) 6026x4024 scale=1,0000 IOP_CS_RAW 18,8106 [opencl memory] device 0: 193989004 bytes (185,0 MB) in use 18,8113 [opencl memory] device 0: 193988992 bytes (185,0 MB) in use 18,8168 [opencl memory] device 0: 96994496 bytes (92,5 MB) in use 18,8170 [dev_pixelpipe] took 0,008 secs (0,000 CPU) [export] processed `temperature' on GPU, blended on GPU 18,9274 [dev_pixelpipe] module `temperature' min: (0,000564) max: (0,453631) [export] 18,9370 [opencl memory] device 0: 0 bytes (0,0 MB) in use 18,9373 pixelpipe process CPU [export] demosaic ( 0/ 0) 6026x4024 scale=1,0000 --> ( 0/ 0) 6026x4025 scale=1,0000 IOP_CS_RAW -> IOP_CS_RGB 19,2015 clip_and_zoom_roi [export] demosaic ( 0/ 0) 6026x4024 scale=1,0000 --> ( 0/ 0) 6026x4025 scale=1,0000 Magick: caught exception 0xC0000005 "Access violation"... ``` ### Commit _No response_ ### Where did you obtain darktable from? self compiled ### darktable version 4.6.0 ### What OS are you using? Windows ### What is the version of your OS? Win 11 ### Describe your system? _No response_ ### Are you using OpenCL GPU in darktable? Yes ### If yes, what is the GPU card and driver? Nvidia 1060, OpenCL on or off doesn't help ### Please provide additional context if applicable. You can attach files too, but might need to rename to .txt or .zip _No response_

Of course 1500 x 1100 (15:11) would also be a simple integer fraction (like how 1920 x 1080 is 16:9).

The interesting part is that 15876 was reported on 14 December; closed on 18 December via PR 15885. Darktable 4.6.0 came out on 21 December. Bug 15951 was reported against 4.6.0 on 24 December, which already had the fix, so I think it was not a duplicate.

15951 was fixed on master, so if your issue is related, you’ll have to wait until 4.6.1 (or whatever release will fix it) comes out – or you can switch to the development version.

priort · January 29, 2024, 9:46pm

I’m on my office PC and the test was version 4.5 + 138… so I built the latest…problem is my tags need reset as it built as 4.5 +1368-gd2299beb8e or something like that but still no issue on my old beast but I think maybe it seems like you might have been getting closer to an AMD issue or specifying some specific dimensions in the scaling…

g-man · January 29, 2024, 10:18pm

The GPU info is then correct. That’s an expensive GPU card.

g-man · January 29, 2024, 10:31pm

PR 15885 was tagged with 4.6.1. It did not make the 4.6 branch.

kmilos · January 29, 2024, 10:47pm

4.6.0 was frozen/tagged before 18 Dec:

kmilos · February 17, 2024, 3:44pm

@JankoK Apologies, I spoke too soon, there was indeed some stuff to fix. Please try again with the just released version 4.6.1.