My Windows install of dt 3.8 is now essentially non-functional after running the ‘purge_non_existent_images’ script, under Windows. Could I ask for some advice on how to recover. Here is the background:
My dt 3.8 installation had multiple raw+jpeg pairs, as well as non-paired jpegs. I identified and physically removed most of the paired jpegs using a Windows Powershell script that I was pointed to, designed for this very purpose. In total, this first jpeg deletion phase worked without problem.
This did not solve the issue of removing the unwanted jpeg image information from my dt database. I had expected skulls to be shown after the jpeg image deletion, but they were not. I wanted to avoid the task of manually identifying those images, which would produce a ‘image is not currently available’ message when an attempt was made to open them. I was advised that either running the ‘darktable_generate_cache.exe’ or the ‘purge_non_existing_images’ script would solve the problem.
I ran the generate_cache program, which took almost 3 hours, and partly fixed the problem. On analysing the results, I realised that there were some ‘raw + jpeg’ pairs where the PowerShell script could not work because the raw and the jpeg images had different filenames (!) - (the toxic result of having to recover from an infamous screw-up in Nikon Image Transfer software from about 12 years ago). These latter jpegs were removed by hand after a visual comparison of the raw and jpeg. In order then to avoid another 3 hours period to regenerate the cache (or so I thought), I chose to run the ‘purge’ script to remove the remaining skulls. The documentation for doing this is relevant to a Linux install only, but there is, on this forum from Feb '20, an article, by GuK, on how to set-up and run this bash script under windows. I followed this article, and finally, after getting sqlite3 installed, running the purge script without and then with the --purge option.
At this point all images, of all types, from all directories within my Pictures directory, were removed from my dt database and library (but not physically removed from the drive). I was left with a completely empty dt install.
My first recovery action was to replace the data.db and library.db files with copies of those files from backup. This did not repopulate my dt install, for reasons I don’t understand - those backups were made before I ran the ‘purge’ script.
My next recovery action was to replace the data.db and library.db files from the latest snapshot version, which were about 6 hours old. This gave me a dt installation with all my image accessible, but with skulls shown for all deleted jpegs, whether from fully paired images. or mages which had been deleted by hand. However, the behaviour of dt was now strange:
- I was unable to select any single image or all images
- invoking the ‘collapse grouped mages’ button caused an individual skull to disappear, but not be deleted - pressing the button again caused the skull to reappear
- the full path information under ‘image information’ was totally misleading: it showed a path to an image which was not visible in the central panel of the lighttable. In fact, positioning the cursor on any grey part of this central panel - any part not occupied by an image - showed this same full path information in the image information area.
- clicking on the ‘remove’ button in ‘selected image(s)’ panel would ask for confirmation and apparently result in the removal of an image, but nothing actually happened.
I decided to run the ‘generate-cache’ program again, but it produced 3 errors for every image as follows: “open-square-bracket image_cache_write_release close-square-bracket sqlite error 8”. Bearing in mind there were now 4 lines of output per image, each taking about 1.5 seconds to show, it was clear that this process would take about 20 hours to complete, so I stopped it.
At that point, I decided to seek help.
I don’t know if it is relevant, but as part of following the article from Feb '20 on how to set up to run the ‘purge’ bash script under windows, I installed the following sqlite3 files in:
c:\Program Files\Git\mingw64\bin
- sqldiff.exe
- sqlite3.exe (version 3.31.1.0)
- sqlite3-analyzer.exe
- sqlite3.def
- sqlite3.dll (version 3.38.2.0)
Any advice?