Collection crashes darktable

Software darktable
1

Hi,

I imported a collection of about 400 Fuji E4 RAW images (.RAF). I was able to go through all images for a first cull. Now I want to edit the remaining images. When I reach about half of the remaining collection darktable crashes while generating previews in the lightroom view. On the command line, I get “Speicherzugriffsfehler darktable” (memory access error).

I tried running darktable with the -d cache argument, then I get a lot of output like “generate mip 3 for image 7279 from scratch”. But there is no indication why it crashes.

I was able to import the directory with the collection in shotwell without crashing, it shows thumbnails for all 400something images.

I am using darktable 4.2.1 on Debian testing.

Any ideas what to try next?

Cheers
Michael

2

You can try to launch with -d all to get more (debug) information.
How many RAM?
Does it make a difference if you try to create the thumbnails/previews via darktable-generate-cache?

3

I have tried it with -d all, producing a log file of 12106 lines for a bit of scrolling in that collection up to the crash. I don’t see anything about the memory access error in the last lines of the log, happy to share it.

I have 16 GB memory, no dedicated video card (Intel chipset, Comet Lake-U GT2). While running darktable-generate-cache the system currently uses about 7,3 GB.

I will report when darktable-generate-cache is finished.

4

darktable-generate-cache crashes in the same region of that collection, again with the message “Speicherzugriffsfehler” (memory access error). The last image mentioned in the logfile is DSCF8970.RAF - is that likely to be the problem file?

5

I would remove this file from the collection. If this is not possible in lighttable (bacause it crashes), quit darktable, remove the file from the folder, relaunch darktable, remove the image from the collection.

6

Here’s the end of the logfile for darktable in live use with that collection, up to the crash. There the last bit seems to be about an earlier picture, DSCF8874.RAF. So I guess darktable stumbles on that image?:

88,504583 [add_job] 21 | 88,504679 load image 7312 mip 3 | queue: 1 | priority: 088,504714 
88,504743 [add_job] found job already in scheduled: 88,504761 load image 7312 mip 3 | queue: 1 | priority: 488,504805 
88,548952 [avif_open] failed to parse `/home/mbelow/Bilder/darktable/20230411_Sambesi/DSCF8874.RAF': BMFF parsing failed
88,555039 [lighttable] expose took 0,0000 sec
88,556138 [add_job] 21 | 88,556201 load image 7312 mip 3 | queue: 1 | priority: 088,556230 
88,556259 [add_job] found job already in scheduled: 88,556283 load image 7312 mip 3 | queue: 1 | priority: 488,556309 
[dev_pixelpipe] module `Weißabgleich' min: (-0,006059) max: (1,972398) [thumbnail]
88,582356 [pixelpipe_cache_get]    thumbnail           CHG       highlights, line  1, age    2, was rawprepare, 407->8483885MB
88,582876 [process CPU]       thumbnail       highlights. IN (   0/   0) 6252x4164 scale=1,00. OUT (1491309/1491309) -1491308x-1491308 scale=1,00, final 1440x1440, backbuf 0x0
88,582998 [dev_pixelpipe] took 0,000 secs (0,000 CPU) [thumbnail] processed `Spitzlicht-Rekonstruktion' on CPU, blended on CPU
[dev_pixelpipe] module `Spitzlicht-Rekonstruktion' min: (340282346638528859811704183484516925440,000000) max: (0,000000) [thumbnail]
88,583079 [process CPU]       thumbnail         demosaic. IN (1491309/1491309) -1491308x-1491308 scale=1,00. OUT (  -1/  -1)    1x   1 scale=-0,00, final 1440x1440, backbuf 0x0
88,583134 [dev_pixelpipe] took 0,000 secs (0,000 CPU) [thumbnail] processed `Entrastern' on CPU, blended on CPU
[dev_pixelpipe] module `Entrastern' min: (0,081250; 0,000000; 0,000000) max: (0,081250; 0,069857; 0,066262) [thumbnail]
88,583198 [process TILE]         thumbnail           ashift. IN (  -1/  -1)    1x   1 scale=-0,00. OUT (   0/   0) 1440x1440 scale=-0,00, final 1440x1440, backbuf 0x0
88,583223 [default_process_tiling_roi] [thumbnail] **** tiling module 'ashift' for image input size 1x1 --> 1440x1440
88,583246 [default_process_tiling_roi] [thumbnail] (1x1) tiles with max dimensions 1440x1440, good 1441x1441, overlap 5->5
88,583279 [default_process_tiling_roi] [thumbnail] process tile (0,0) size 1x1 at origin [-1,-1]
7

I think the logfile for darktable-generate-cache contains just success reports, even with the option -d all. The last line in the darktable-generate-cache output is:

image 7311/7818 (93,51%) (id:7312, file=DSCF8970.RAF)

So I will try moving DSCF8874.RAF out of the way.

8

Moving 8874 didn’t help, but I am getting interesting new messages from darktable on the command line. The time before last it said:

Gdk-WARNING **: 20:09:49.399: Tried to map a popup with a non-top most parent

This time it said:

wait time 0,104861s
try- wait time 0,107986s
wait time 0,113434s
9

Hi @mic and @pehar,

Michael wrote

400 Fuji E4 RAW images (.RAF)

Each X-E4 .RAF is 54.84 MB large…
Multiplied by 400 images equals 21936 MB.

Might it be a RAM and/or swap-problem?

Have fun!
Claes in Lund, Sweden

10

I would remove this file first. If this does not help, I would remove the image with id 7313, probably …8971…
If you are able to work with sqlite3 you can look into the database library.db to find the filename corresponding with id 7313.
If I remember correctly (not on my computer at the moment) the id is also shown in lighttable → image parameters. But if dt crashes in lighttable moving to the critical image, it might be impossible to find it this way.

11

All right, looks like removing DSCF8970.RAF has done it. Funny enough, re-importing the RAF without the XMP works fine, while re-importing the RAF with the XMP crashes darktable.

I have already reported this issue as a Debian bug, https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1034509 . Will add the details about the broken XMP crashing darktable there.

Should I send a separate bug report to darktable?

12

This is what I expected. Have you already edited this image? Is the xmp created by darktable (not modified by an other application)? If yes : it might be useful to create a darktable issue at github providing the critical image and the corresponding xmp.
You can also reimport the image and do the (identical) processing again, checking step by step which moule is causing the crash.

13

Hi Claes, thanks! I think I found a more localised solution - removing one file seems better than exchanging RAM or my SSD :slight_smile:

1 Like
14

I have filed an issue in Github: XMP file crashes darktable, segmentation fault · Issue #14241 · darktable-org/darktable · GitHub