I’m experiencing some weird behavior with DT 3.7 after my Windows 10 machine crashed and corrupted the database and library files
Unable to recover the files, I renamed the most recent snapshots, which restored the library, however:
(a) Clicking an image in Lightroom won’t select the image
(b) Double clicking an image takes me to darktable, but the program fails to invoke any of the initial modules, and history only displays “original image”
(c) Images won’t duplicate
Starting fresh with an empty folder restores full functionality, but copying the renamed snapshots will revert to the earlier behavior
So I suspect there problem is within the snapshots. Is there anything else I should be doing other than renaming, or is there a command line or other action to take?
Just to make sure: Did Windows crash and did it do the, possibly more global, corrupting or did darktable crash and did it corrupt its own files?
If I look at the overall picture you draw about what’s happening after the crash I’m getting the impression that Windows crashed and corrupted parts of your machine.
Especially point b is “interesting”: When you load an image in darktable the associated sidecar, which holds the edits, is loaded. As far as I know you don’t need a populated data.db and/or library.db to make this happen (you can install a clean darktable and still load the edited RAWs as long as the correct sidecar naming is adhered to).
Looks a bit like your filesystem might be corrupted or at least in need of repair. Do you have any issues with any of your other programs running on that machine after the crash?
I’m not a Windows man so I can’t really help you with that, but maybe if you give a bit more info someone might be able to say something definitive about all this and/or point you in a certain helpful direction.
EDIT: Just thought about this one: Have you tried cleaning darktable’s cash directory?
I moved to using a temp db in memory for various reasons …multiple machines running DT etc etc I would find myself with database issues…so I moved to that. So I am essentially always loading images/edits using side cars with no issues…
So Dave I think if you run DT with configdir switch pointing to a new directory then DT should load images using your sidecar files. Or you could try using a virtual library with the memory command line parameter…Either way if this works then I think DT is okay and it’s your databases…if there are further problems then it could be wider corruption
@Jade_NL and @priort, these are all good insights, but I found I can recreate the problem in 3.6.1, which isn’t corrupted.
If I delete the 3.6.1 data and library files and replace them by renaming data.db-snp-XXXXXXXXXX to data.db and library.db-snp-XXXXXXX to library.db (question: is that correct?), then the earlier library is restored, but all the problems I described will occur.
Restoring the original data.db and library.db files brings everything back to normal operating conditions. So something is happening with the snp files.
It’s my understanding the snp files exist as a backup in case the data and library files get corrupted, so either there’s an issue within the snapshot files, or I’m doing something wrong.
I’m not saying that it doesn’t prove anything but you can’t go back from 3.7 to 3.6.1. This due to database changes that took place which aren’t compatible and/or recognized when downgrading.
About renaming the snp files: Yes, you should rename them to data.db or library.db depending on the file. On Linux you also need to set the correct permissions: They both need to be readable and writeable by the user that started darktable.
Most people don’t do this but just in case: Have you tried restoring from your system backup?
Before I start with things you could try: I’m assuming that you have made a backup of, at least, these files:
darktablerc
data.db + snp versions
library.db + snp versions
The following should be done from a terminal. darktable might spit out some useful info about what’s going on. Do keep an eye on the messages shown.
Quit darktable, empty your darktable cache, restart darktable: Does the problem remain.
Quit darktable, Remove darktablerc, data.db and library.db Start darktable.
Does darktable work as expected? If it does:
quit, restore backed-up darktablerc, start. Does it still work? If it does:
quit, restore backed-up, data.db, start. Does it still work? If it does:
quit, restore library.db, start. If the above steps worked then this one shouldn’t
PS: I assume the original files, not the snp versions in the above. You can expand this test by checking the snps as well.
You should have an idea which file (files?) is the culprit. I would remove it and don’t look back. Not having one of these (or all of them for that matter) might be inconvenient, but your edits are still available as sidecars so nothing of real importance is lost.
You might want to run darktable-generate-cache to create a fresh and up-to-date thumbnail cache. If I assume a default installation then you don’t need to set any of the option.
BTW: Depending on preference settings 1 or more snapshots are created and kept for a certain amount of time. Have you tried one of the older ones?
So, I tried all your steps and have pretty much determined that the errors occur after I attempt to restore from the snapshots, plus I found that DT crashes if I attempt to add new images (I have backtraces). Deleting those files and starting fresh restores all functionality without issue.
At this point I’m focused on gathering information to submit this as a potential bug report via GitHub.
