Strategies for using darktable without its DAM feature

For me, that wouldn’t really work, as I do use some of the search capabilities of darktable, over several (many) film rolls/folders, and that needs the database.

It might work. I do go back and forth, but not more than a few times/day, and I’m not bothered by the start-up delay. So I prefer not having them both running.

As for the keyword setup, I set that up a while ago and forgot the details…
The issue I had before changing the digikam setup was that I’d get elements of hierarchical tags as separate tags in my tree. Which made working with the tag tree unnecessarily complicated.
Iirc, darktable uses the Xmp.lr.hierarchicalsubject tag , and the Xmp.dc.subject tag. Digikam reads the first item it understands, so you want to make sure the Xmp.lr.hierarchicalsubject tag is tried before the Xmp.dc.subject tag. For writing, you may even want to disable the Xmp.dc.subject tag (but how you want to set that up exactly depends on what other programs have to understand your metadata).

That’s what I have doing in the past couple years:

  • culling, rating, tagging in DigiKam
  • editing in Darktable.

Make sure to set Read and Write to XMP-files in Digikam

As a precaution, don’t run Digikam and Darktable at the same time.
When opening Digikam, I always do a rescan on the folder I’m currently working in Darktable. You can set DigiKam to automatically rescan your collection on startup. As my collection is very large and lives partially on a NAS, this is slow an tedious.

Thanks. Have you added all your photos as collections in darktable too? Or are they just added to collections as you open them from digiKam? May I ask how big your collection is? Are you experiencing any slowdowns that you think are a results of the collection size? My whole collection is on a NAS.

The answer is a bit longer than this. I keep different darktable databases on my system. I keep a database for every year. I keep multiple shortcuts to darktable with the --library parameter pointing to the database.

Digikam contains my entire collection; living partially on my workstation, partially on my NAS. All together this is more than 2TB of data, over 100.000 files. (More than 20 years of images).

Darktable in particular the darktable database only contains the files of that year.
2023.db, 2022.db etc.

I don’t know if this is recommended, nor do I know what the performance impact is when having only 1 database with a lot of files.

Working from the NAS is slow, but that has to do with the speed of my nas and network throughput.

Geeqie does support jpeg/raw pairs, and I use them (I shoot raw+jpeg because I like wasting SD card space :smiley: ). Geeqie will delete the pair together when culling, if desired.

In darktablerc, I changed database=library.db to database=:memory:
That causes dt to not use a persistent database.
The Geeqie plugin for dt invokes dt with the input image file name. That causes dt to open into the darkroom with the image ready to go. If you do switch to the lighttable, under collections you will see the one file all by itself:
image
If you then exit and re-enter dt, that file will no longer show up in collections, because there was no database. So, like Terry said, you can just ignore the collection because it’s transient.

1 Like

I would like to add a correction here: my entire photo collections lives on my NAS. Only the last 2 years are synchronized to my workstation. I use Syncthing for this.

If you do not use the database, what other consequences does it have? I seem to recall that styles and others things are stored in the database also. Are they lost if I don’t use the database?

As I use external hard drives that may or may not be connected, which seems to leave a lot of entries in the collections module shown with strike through text. Can’t seem to get rid of those.

Image info is in library.db. Presets and styles are in data.db, which is separate from library.db, so you should not lose any.

You should not see those with database=:memory: because there is no database being accessed to load historical collections information.

2 Likes

Hello mikae1,

Quite late I am answering to your original post at the top. And funny, I read today your quoting of my reddit post about my parallel use of DK and DT.

I am still very happy with separation of DK-DAM and DT-Editing. May be also, because I have 67,060 JPG files to manage and only 12,461 RAW files to edit. I do not need previews of my raw files, because I export always my edits as JPG files. In addition I create with automated batch jobs dowscaled (to 1080 pix height) and sharpened JPGs for fast viewing on PC/TV and sharing via e-mail or web gallery. I do not need and want all these JPGs in DT, but of course in DK. In DK I am only tagging the master files (RAW or JPG). If I search a photo, I find either the master JPG or the master RAW. In case of RAW file I have to look for the RAW-to-JPG-Export, which is located in the same DK-group or due to the same leading file name directly next in file manager.

Thanks a lot, that could be helpful.

I really can’t understand by what standards a design that opens a program to a wall of skulls could be considered a brilliant idea.


As I wrote, I use external hard drives that may or may not be connected, and Darktable doesn’t seem to be able to handle that.
A simple filebrowser like Geeqie’s which is blazingly fast for browsing NEF files would be a wast improvement over the wall-o-skulls.
Alternatively a simple check to see if last used directory is still available would be nice, so I wouldn’t have to see the list of collections in over strike font …

The skull means “file not available.” Since you use an external disk, maybe you want to use the local copies feature?

It does “handle it” by telling you it can’t find your files.

What would geeqie show you if the disk containing the files isn’t connected?

And what would you have dt do if the disk isn’t available?

Well Geeqie does not attempt to show me a list of directories I used last time around and which may or may not have been moved/deleted. Geeqie just helps me find the files I want to see now.

Fair enough that you want Darktable to import and maintain a list of directories/files you worked on last week, but it should be able to handle other types of workflow. Like for example how most other programs handle files …
Most of all since importing complete directories of files into Darktable still does not give it a speed advantage in displaying files over Geeqie or other programs.

Yeah I know what the skulls mean, but I really don’t need a program patronizing me by using two thirds of my display to get that point through. :wink:

You can run darktable with --library :memory: and it won’t store any images to the database.

Its about convienence, not speed. I like to see what I edited last.

Most programs (including dt, depending on your settings) show you the raw file’s embedded thumbnail, that’s why they’re fast, they’re not processing any of the raw data.

Its not patronizing to know that images aren’t available. There is a feature to make them a available (local copy) and then you can filter your collection down to what is available locally.

I def went an indicator of when images in my library are not available, as that likely signals a serious underlying problem.

Just different use cases/user preferences. As I read somewhere, in relation to photography I think, trying to please everyone leads to nothing but compromises… I suspect this applies to software too. :wink:

Different programs, different aims, different approaches. Somehow I guess you don’t like the editing options Geeqie provides (otherwise, why use dt?).

Darktable made certain design choices, one of which is assuming as default that images are on a permanently available medium. A way to change that default behaviour was given (“darktable --library :memory:” or change the corresponding entry in the configuration file).

When you keep the default behaviour, you are warned when images are not available as you should be: dt comes back to the film roll where you exited the program. A film roll is not just a synonym for a folder, but can be the result of a more or less complex search you set up. That film roll was not empty when you last exited dt. Now it is, so dt tells you “I have those images in my database, but they are not where I was told to expect them”.

Again, no. Copy-local exists. You can use external or network disks as your primary storage for raw files. OP also had something happen to their cache, or else they’d see thumbnails. I used an external disk for several years and didn’t run into this.

There is a script to remove non-existing images from the database.

purge_non_existing_images.sh

No more skulls. aesthetically displeasure problem gone. (I don’t like them much either)

I found out about it, I guess, from this forum. It is covered in the manual in the “Program invocation” chapter.

Here is my simple workflow involving Darktable/Ansel and digiKam (only for DAM). My main folders are:

Photo\1 Process - I put unprocessed files there (mostly RAWs, but also unprocessed TIFFs from a scanner), organized in Year\Month-Day-Event folder structure. This folder is for Darktable.

Photo\2 Export - “Final” JPGs exported from Darktable (or Affinity etc.) - this folder is for digiKam only.

I export files from Darktable using the path below:
$(FILE_FOLDER/1 Process/2 Export)/$(FILE_NAME)
This simply replaces “1 Process” with “2 Export” in the folder name. This way both folder have the same file structure inside.
So, for example “1 Process\2023\08 25 Trekking\IMG_2995.cr3” is exported as “2 Export\2023\08 25 Trekking\IMG_2995.jpg”.

After exporting a photo I add tags in Digikam (inluding face recognition), all data is saved to XMP files. I can run both programs at the same time - they don’t overwrite each other’s data.

The best part - whenever I decide a photo needs some tweaks (which happens VERY often - I’m still learning :slight_smile: ), I can re-export it from DarkTable (I set “On conflict: overwrite”). The XMP file created with digiKam is preserved, only the picture is updated.

Bonus: if I decide to try another DAM solution, I won’t be held back by the digiKam’s database because all tags are stored inside sidecar files.

I hope this makes at least some sense :slight_smile:

Perfect sense and I have almost exactly the same workflow myself.

The only issue is re-tagging after tweaking. Each new export from DT needs to be retagged in DigiKam, but if the photo was worth redoing, its only a small hardship.

My experience is different. The few times I had to add keywords in dt, digikam read the new keywords.
But I do remember I had to change the order in which digikam used xmp tags: in the digikam settings for metadata, you can set xml tags will be used for reading and writing, and in which order they will be used for reading.