dt: recognising newly imported xmps for duplicates on start-up

I run dt under Linux and Win 10 , on separate computers, synchronising the images and side-cars (using FreeFile Sync). I commonly create image duplicates while doing post processing on either computer; the duplicate XMPs are then synced (valid word ?) to the other computer, making sure dt is not running. The duplicated images do not appear in lighttable when I start dt on that other computer. I have previously been advised, on this forum, that the setting ‘look for updated XMP files on startup’ (in preferences, storage) does exactly that: look for updated XMPs, not new ones. I understand that.

To make dt show the newly added duplicate, in lighttable I ‘remove’ the image for which a duplicate has just been added via synchronisation and then add that image back in again via ‘import, add to library’. This invariably picks up the duplicate XMP as well as the original image and XMP. It’s a simple process to follow, but I’m wondering if there is a more direct way?

As far as I know, that’s the only way to add existing duplicates to the database. I’m not sure you need to remove the image first, though.

2 Likes

This post is intended as a cautionary tale for the (probably tiny) audience of dt users who maintain separate, but synchronised, instances of darktable, as I have described above, under Linux and Windows. The caution is that the dt implementation of ‘picking up’ newly created side-car files can lead to discrepancies between the two instances of the dt library.

The discrepancies are a result of the fact that for a newly added image no sidecar file is created until the image has been opened and edited in the darkroom (*). This is especially a potential source of discrepancy between the number of files on disk and the number of images reported in lighttable, when a duplicate of an un-edited file is created. Lighttable then shows the number of images in the collection as +1 to the number of image (not sidecar) files on disk.

Synchronising between Linux and Windows at this stage, using a tool based on comparing file size and date, will lead to a difference in the image count shown in lighttable between Linux and Windows, even though synchronisation has completed correctly. This can be confusing and leads to doubt about the integrity of either or both instances of dt. In fact there is no integrity problem.

Further more, if the duplicate, rather than the original, of a newly added image is edited in darkroom and then synchronisation is performed, the image count shown between the two instances of lighttable will still differ by 1. In addition, the ‘synchronised to’ instance of lighttable will show an image which is at version 1 only and, apparently, for which no version 0 available. To have an image with no version 0 but a version1 can be very confusing.

Furthermore, as per my original post, a newly created sidecar is not picked up by a re-start of dt on the synced-to instance of dt. I now understand that it is not necessary to ‘remove’ the image first, as I previously suggested and which rvietor questioned. However the updated sidecar file is picked up only when that specific image file, and only that image file, is added again using the ‘import, add to library’ process. Selecting all the images in that folder, which is quick to do, does not result in the updated sidecar being ‘discovered’.

In summary, these discrepancies and potential confusions when using synchronisation as described, are avoided by ensuring that all images have been edited in ‘synced-from’ darkroom first.

(*) I think I have read, or been advised, that it is necessary only to open an image in darkroom for a sidecar file to be created. That has not been my experience – I find that it is necessary to invoke a darkroom module on an image before the sidecar is created, even if a duplicate has been created on an un-edited image.

Depends on your settings.
https://darktable-org.github.io/dtdocs/en/preferences-settings/storage/#xmp-sidecar-files

A very useful reminder - thanks. Having read the info. at the linked location, and tried out all three options for myself, I am now inclined to think that I should be using the ‘on import’ option rather than ‘after edit’ which, up to now, I have convinced myself as being the most suitable option for my workflow.

That points to a general problem: having chosen an option on a setting in preferences, we rarely go back, at a later time, to re-affirm that our choice is still the most appropriate one. In this example, I had completely forgotten about the 2 other options. So, thanks again for the reminder.

2 Likes