How can I keep the thumbnails from session to session?

Software darktable
1

My images are stored on an external drive, and I prefer to keep them separated from my C: drive (my PC), which means that I edit and manage all images referring the external drive.

The first time a film roll (collection) is loaded, it takes from a few to a lot of seconds to create the thumbnails (embedded jpg), depending of the file size obviously. This is a bit strange to an technical numb person, as I work on a fast PC.
When a number of film rolls are loaded, the shift from one collection to another is without any delay of importance.
But if Darktable is restarted, the boring creation of thumbnails starts all over.

Is there a way to keep the generated thumbnails alive from session to session?

2

Thumbnails are cached, see manual darktable 4.6 user manual - thumbnails for details.

darktable will keep all thumbnails on disk if the corresponding disk backend option is activated in preferences > processing > cpu/gpu/memory.

3

These are

cpu-gpu-memory settings
cpu-gpu-memory settings771Ɨ246 7.19 KB

the settings - with the problem as described.

4

Which dt version ?

Maybe a permission issue on Windows as clearly the thumbnail should not be regenerated. Look to see of the cache is storing the thumbs.

Canā€™t help more as Iā€™m not using Windows.

5

The version is 4.2.1.
I donā€™t know where to look for the cache? Sorry to be a plain ignorant user.

6

Can you describe your process in more detail? Where are the images? NAS? cloud? Are you mounting a drive? Are you using a flag (Eg. -config) when you start darktable?

7

I am using a USB connected external hard drive. Nothing exotic. Itā€™s plain normal I am afraid.

8

Are you mounting the drive with the same windows drive letter every time?

9

Yes my man.

10

When darktable generates the thumbnails, they remain in the .cache folder (windows is a very long path that is in the FAQ).

Are you importing the images again? Do you get skulls? Are you having to point the images to the drive via the search filmroll?

11

I am not sure, what you mean. I donā€™t get skulls. The load is automated, I donā€™t do anything else than select a film roll from the list, but when I do it ā€˜the following dayā€™ the thumbnails are generated again.

12

AFAIK the location depends on Windows version. I remember something like C:\Users\[username]\AppData\Local\Microsoft\Windows\INetCache\darktable\.cache or similar. Please search the forum and look at faq | darktable.
You can also use the commandline tool darktable-generate-cache to force the creation of all missing thumbnails/previews (darktable 4.6 user manual - darktable-generate-cache). The tool creates an output line for every processed images, possibly error messages if something wents wrong.
I do not use Windows, unfortunately can not help further.

13

The darktable-generate-cache rutine ran for a loā€¦ng time. But this cache survives a cold start of the PC.
Apparently the thumbnails generated from just opening a collection were of a more temporary nature. The InetCache is forbidden area. The path says ā€¦ Windows\InetCache\darktable\mipmaps ā€¦ but I am not allowed past Windows, and thatā€™s fine, if this cache stays around.
Thank you all for suggestions.

14

You are most probably allowed, but the directory (folder) is hidden. In your Explorer:

image
image1191Ɨ213 20.6 KB

image

First, select Show hidden files, folders and drives, and uncheck Hide protected operating system files, and then click Apply to Folders.

15

OR do like the FAQ says, type the path in windows explorer.

16

Thank you, Kofa - Untagging the ā€˜Hide protected os filses ā€¦ā€™ allows me to see all thumbnails.

17

Yes, g-man, if I re-invoke the tag in ā€˜Hide protected os files ā€¦ā€™ and write the path in explorer - I get access to the thumbnails too. Nice to know.

18

One day later, the thumbnails are gone - or at least not recognized by Darktable.

On a fairly fast PC the routine ā€˜darktable-generate-cacheā€™ takes about 15 minutes to create 300 thumbnails, which feels long these days - and as you can read below - the routine and Darktable (apparently) canā€™t work simultaneously.

The routine will not start if Darktable runs - db locked.

If Darktable is run with the routine running, this error occurs:
[image_cache_write_release] sqlite3 error 5 (this stops as soon as Darktable is shut down)

(no script in the script manager seems to manage this)

Apparently the thumbnail cache will disappear over a cold night - but to save time and resources could it be kept alive?

19

Thatā€™s not dt issue, the cache is not removed by it. So either a bug which is specific to Windows or a plain Windows issue.

1 Like
20

But why do those thumbnails disappear?
For me (dt 4.2.1 under Linux) they are kept overnight (and even longer, quite a few were last modified 2 months ago or more). As I donā€™t use darktable-generate-cache, those thumbnails were generated by darktable.

And, are the thumbnails actually gone, or not seen/recognised by dt?

That you cannot run darktable and darktable-generate-cache simultanuously is understandable: they both need read/write access to the database, so have them both running risks database corruption (SQLite isnā€™t designed for that use) or data loss. That you are allowed to start dt with darktable-generate-cache could be a bug?

That ā€œsqlite3 error 5ā€ indicates a database lock. If thatā€™s thrown by darktable-generate-cache, itā€™s very likely that the newly generated thumbnail isnā€™t registered in the dt database (but the image will exist on disk). Such thumbnails wonā€™t then be seen by darktable.

Iā€™d suggest you make sure not to run darktable and darktable-generate-cache simultanuously. Also, you can try to delete the current cache, then see what happens (perhaps use only dt on a small film roll). If your newly generated thumbnails survive, you have an idea about the causeā€¦

Another possible cause for ā€œdisappearingā€ thumbnails could be a change in thumbnail size. Thumbnails can be generated at different sizes, by default (i.e. without parameters, darktable-generate-cache generates 3 sizes (ā€œ0ā€, ā€œ1ā€, ā€œ2ā€, for me those have a height of 110, 225, and 450 pixels, resp.), but the size can be up to ā€œ8ā€ (full size?).

And of course, it might conceivably be possible that something on your system clears caches overnight, or when they go over a certain size, or when free disk space drops below a limit, orā€¦ Point being that a cache can be volatile.