Organizing photos while preserving XMP sidecars

I have photos I’ve edited with Darktable that have produced xmp side car files.

Like:

./Taiwan/DSC01929.ARW
./Taiwan/DSC01929.ARW.xmp

I am trying to organize my photos into dated folders like, “2025/01/20/Taiwan/DSC01929.ARW”

I need the xmp files to go with them to preserve those edits. So far I haven’t been able to get exiftool or Digikam to recognize and bring those with the images.

Any help is appreciated.

EDIT
If using Digikam, pay particular attention to the metadata options:

The default filename for sidecars is automatically set to filename.ext.xmp. For example, image1.dng will have a sidecar file named image1.dng.xmp. If the Sidecar file names are compatible with commercial programs option is enabled, digiKam will instead create the XMP sidecar files with a compatible file name (image1.xmp) used by many commercial programs.

XnView MP has an excellent search engine which can find files like that, present them and only them as a ‘contact sheet’ from which you can save them anywhere you like.

Thank you @cedric for the recommendation. I installed it and it does indeed have a great search that considers the xmp file, but not any way to organize them into folders, only batch rename the files.

I think exiftool can do what I’m trying to do, I just haven’t found the right combination of options.

1 Like

There is a move or copy option in DT’s lighttable view which is very handy for moving a small number of images to reorganise them, but this is probably unsuited to handling a large numer of images from various dates.

1 Like

Or have a look at digikam, it allows you to create albums and move images around (albums equate subdirectories on disk in digikam). XMP files are moved with the images, in principle: I do not know what happens if you created duplicates in darktable…

But be careful with any way to move images around outside a program that doesn’t use sidecar files. If you use e.g. a script or a file manager, it is your responsability to keep sidecars with their images.
This is of course not an issue when you move complete directories around, where images and sidecars always stay together.

In any case, you’ll have to re-import your whole collection again in dt, unless you use dt to do the moving.

I use the following bash script (you did not state your operating system, but you can even get it for Windows, or probably ask an LLM to convert it to Powershell) to rename files.

for i in *NEF *RW2 *JPG *jpg ; do echo $i exiv2 -g Exif.Photo.DateTimeDigitized $i 2>/dev/null ; done | grep DateTimeDigitized |gawk '{print "mv", $1, $5 "-" $6 "_" $1, "; mv", $1 ".xmp", $5 "-" $6 "_" $1 ".xmp"}'|sed 's/:/-/g'|bash

This takes the images with the specified extensions, extracts the date, and creates mv (move) commands to rename the files. You could alter it to create a directory instead, and move the files there. Currently, it creates commands like this:

mv P1070308.RW2 2025-05-01-12-54-12-P1070308.RW2 ; mv P1070308.RW2.xmp 2025-05-01-12-54-12-P1070308.RW2.xmp
mv P1070309.RW2 2025-05-01-13-07-07-P1070309.RW2 ; mv P1070309.RW2.xmp 2025-05-01-13-07-07-P1070309.RW2.xmp
mv P1070310.RW2 2025-05-01-13-09-22-P1070310.RW2 ; mv P1070310.RW2.xmp 2025-05-01-13-09-22-P1070310.RW2.xmp
mv P1070311.RW2 2025-05-01-13-16-48-P1070311.RW2 ; mv P1070311.RW2.xmp 2025-05-01-13-16-48-P1070311.RW2.xmp
mv P1070312.RW2 2025-05-01-13-18-48-P1070312.RW2 ; mv P1070312.RW2.xmp 2025-05-01-13-18-48-P1070312.RW2.xmp
mv P1070313.RW2 2025-05-01-13-19-37-P1070313.RW2 ; mv P1070313.RW2.xmp 2025-05-01-13-19-37-P1070313.RW2.xmp

(Edit: sent scripts in a message.)

1 Like

I never got deep into substitution in variables, but just to throw in the idea, this is useful when dealing, for instance, with jpg and raw, etc file pairs. For instance, this is my first-cull script before importing what’s left into dt.

for i in *.ARW
do
    fname=${i%.ARW}
    if [ ! -f ${fname}.JPG ]
    then
    	rm ${fname}.*
    	fi
      done

I’ve deleted jpgs, so it looks for ARWs that don’t have a JPG (culled) and removes all files with that name and any extension.

Your script goes deep with the exiv stuff. I’ve never used that in scripts. Maybe one day :slight_smile:

(Back in the (work) day, I was a big fan and user of awk)

I have tried Digikam and it does a great job to organize the photos, but it doesn’t move the xmp with the ARW files.

Edit though I hit reply to @rvietor. Still getting the hang of this discuss platform

Have you tried the scripts I sent you in a message?

That’s surprising…

Are you using the XMP naming scheme (<name>.ARW.xmp) used by both darktable and digikam? In that case, it would not only be surprising, but a serious bug.

If the failure to move sidecars concerns duplicates from darktable (<name>_<n>.ARW.xmp) I’m less surprised (I more or less expect those to be ignored, as digikam doesn’t use such sidecars).

1 Like

You have to set digikam to use the non standard darktable xmp naming convention. I don’t think is uses it by default?

2 Likes

Thank you both!

According to Dikigam’s manual:

The default filename for sidecars is automatically set to filename.ext.xmp. For example, image1.dng will have a sidecar file named image1.dng.xmp. If the Sidecar file names are compatible with commercial programs option is enabled, digiKam will instead create the XMP sidecar files with a compatible file name (image1.xmp) used by many commercial programs.

The left behind xmp files are named like image1.xmp. I will either need to rename them (probably some exiftool conversion to ensure the correct raw format extension is used) or toggle that checkbox on and deal with the image1.ARW.xmp files, which there are fewer of.

Thanks again!

Goodness! If it were only that simple. The problem I’m running into now is it doesn’t keep them on import if you change the directory structure.

Import > Add Images
Under Auto-creation of Albums
Select Date-based sub-albums
Date Format: Custom, yyyy/MM/dd.

XMP tab is populated with the Darktable info. After import, no data in the XMP tab and xmp files aren’t copied alongside their RAW counterparts.

So you have both filename.ext.xmp files and filename.xmp files?

If that’s your issue it’s pretty damn hard to solve it without loosing data. Unfortunately it’s a design decision made by DT devs and they won’t budge. All users have to know about the issue and how to avoid it before using DT otherwise they will end up with multiple .xmp files that you can’t realistically merge.

Erm, I don’t have that option. I can pick either “Add to library” or “Copy & import”.

“Add to library” does just that, add existing images to the library without any moving or copying. But it does read existing sidecar files.

“Copy & import” does not read existing sidecars, according to the manual:

Which is why the suggestions are to move the files within darktable, where the files are already in the darktable database, or to use an external tool and accept the responsability to move image and sidecar together…

I’m using Arch Linux. Maybe the layout is different depending on the OS?

I have finally found a solution to organizing my collection, while preserving the XMP sidecars. It was very simple after all.

exiftool '-Directory<DateTimeOriginal' -d "./Library/%Y/%m/%d" -r -ext arw -ext raf -ext nef -ext tif -ext dng -ext heic -ext jpg -ext jpeg -ext png -ext xmp ./unsorted-photos

This handles all the formats I care about and importantly, -ext xmp includes the metadata and edits from Darktable.

3 Likes

Pardon the late reply.

It can be done:

In the browse view, select the files you want to move to any folder, then right-click and select ‘Copy To …’. In the ensuing dialog you can enter an existing folder or create a new one.