Darktable does not write into sidecars generated by digiKam

Hi all,

after several years of smoothly working with digikam and darktable I noticed (by chance) that darktable is not writing the edits to xmp sidecars generated in digikam anymore. When the sidecar was generated by darktable, everything works fine as before, even with multiple changes in both. The issue obvioulsy started in April 2025 and after a few hours of fiddling and downgrading I am relatively sure that the problem has to do with the xml structure. Darktable generated sidecars start with <?xml version="1.0" encoding="UTF-8"?> while digikam begins with <?xpacket begin="" id="" ?> (since April 25, before they started with <?xml> too).

My workaround is to import new images to darktable first, let it write the sidecars and open the images in digikam after. If there is no other solution, I could live with that workaround in the future too, but (as probably the most of you) I would prefer to check and tag images first before editing them.

Downgrading to older digikam versions (AppImages in my case) is somewhat strange: I tried it until back to 8.3.0 (March 24) and all older versions surprisingly start the sidecar with <?xpacket> too. As mentioned above, they didn’t do so before April 25: I have thousands of files generated by digikam versions 8.3.0 to 8.5.0 before April 25 starting with <?xml>.

So at the moment I’m quite overwhelmed with this issue and would appreciate any hints what to do next.

Addendum: Using ‘write sidedar files’ in darktable’s history stack module is not working either, when the xmp starts with <xpacket>.- Since darktable does not try to write a new sidecar file, it obviously detects that there is one avalaible, but doesn’t seem to accept it and so does nothing at all.

Addendum 2: Meanwhile i doubt my assumptions were correct. Starting the sidecar with <?xpacket> does not seem to be the core problem, as darktable overwrites it with <?xml> as long as the orginal sidecar was not generated by digikam. Opening a file edited in darktable with digikam changes the header back to <?xpacket> and darktabe changes it to <?xml> after. You can repeat that as often as you like, if just the original file was generated by darktable. So the problem is rather something with the initial creation of the sidecar by digikam, but not the xml structure. Sorry for the confusion!

There is a setting in Digikam about what extension to use for sidecar files. Maybe that is the culprit?

There is a setting in Digikam about what extension to use for sidecar files

Thanks, but the settings are the same since years: I use basename.ext.xmp and leave ‘sidecar file names are compatible with commercial programs’ unchecked, as it would cause the file name to be basename.xmp instead and thus (as far as I remember) the sidecar would be ignored by darktable at all. So I tend to exclude a setttings issue.

darn

1 Like

Can you post a sample XMP? Do you have darktable set to look for updated XMPs on startup?

@paperdigits Okay, thanks, here are some samples of the same image.

  • dk_only: created in digikam, darktable will not write to it
  • dt_dk: created in darktable, written by dgikam after, darktable will write to it
  • dt_only: created in darktable with no further process

Yes, darktable checks for changes on startup. No problem with that.

img_dk_only.dng.xmp (4.8 KB)
img_dt_dk.dng.xmp (12.2 KB)
img_dt_only.dng.xmp (9.1 KB)

And here is the image, just in case:

img.dng (73.2 MB)

the xpacket processing instruction is weird, and with the whole XMP data wrapped in a processing instruction, it is not surprising that dt doesn’t read it, as processing instructions are not XML and the parser should just pass processing instruction along to the processor.

@Michmill do you know anything about this xpacket wrapper?

@paperdigits Just for the records: I tested AppImage versions 8.70, 8.6.0, 8.5.0, 8.4.0 and 8.3.0. All of them show the same behaviour. Strangely this definitely didn’t happen when versions <= 8.5.0 were up to date. So maybe some other component is playing along.

Well, I also use both digikam and darktable, importing files first into digikam, adding keywords and descriptions, and then importing the files into darktable (through “add to library”). Not only does darktable read the digikam-generated sidecars, it also writes the edits back into the sidecars
 And digikam is able to read back those sidecars. That is with digikam 8.7.0 and darktable 5.2.0.

To add to the confusion:

  • after digikam, and before import in dt, the sidecars start with <?xpacket begin="ï»ż" id="W5M0MpCehiHzreSzNTczkc9d"?>
  • after edits with darktable, the sidecars start with <?xml version="1.0" encoding="UTF-8"?>

Oh, and wikipedia, gives an example XML document for serialized XMP metadata in a JPEG photo, which starts with <?xpacket

In the dt and dk headers, is there a declaration and is it significant?

Rats, I meant <doctype> declaration and today my stupid forum editor is not working.

No, there is no doctype.

Thanks. Another possibility, according to Microsoft, is that the offending sidecar file are, or have been caused to be “not well-formed”:

You may hear someone from your IT department mention “well-formed” XML. A well-formed XML file conforms to a set of very strict rules that govern XML. If a file doesn’t conform to those rules, XML stops working. For example, in the previous code sample, every opening tag has a closing tag, so the sample adheres to one of the rules for being well-formed. If you remove a tag and try to open that file in one of the Office programs, you will see an error message, and the program will stop you from using the file.

You don’t necessarily need to know the rules for creating well-formed XML (though they are easy to understand), but you do need to remember that you can share XML data among programs and systems only if that data is well-formed. If you can’t open an XML file, chances are that file isn’t well-formed.

That is not the issue.

OK, just trying to help.

And I’m just giving you an answer. I use XML every day at my job and working it with is my primary responsibility.

So, if I understand correctly, dt and dk XML files are always well-formed and follow all the pertinent rules for being well-formed.

Correct.

@rvietor Exactly the same happened to me too, but unfortunately doesn’t anymore. Do you use the apps on Linux or Mac/Win? And if on Linux: You use AppImages or other packages? Which distribution?

I’m still trying to find where this long-functioning workflow broke off.

Linux, binary packages or self-compiled. No appimage.