missing features: film roll and sidecars?

It can read and write them.

This is true even if one were to hypothetically never use another photographic program on their desktop computer. If you upload your tagged photos to Flickr, for example, Flickr automatically reads the XMP tags and incorporates them on the photo’s page, meaning they show up in keyword searches etc. That’s the power of an industry standard.

1 Like

That’s embedded XMP though, not sidecar XMP.

I can get behind that, though, as I tag my manual lens models on Flickr.

If you’re going to go to all the work of supporting embedded XMP, then sidecar XMP is trivial.

Sidecars exist because writing metadata to reverse engineered files like RAW files is foolish and dangerous. JPEG is an ISO standard so no worries there — embed the XMP.

Sidecar XMP for RAW becomes embedded XMP in the exported JPEG :slight_smile:

Looking for sidecars was one of the first things I did when testing out your nice software.

Without them it is impossible for me to use it.
Not that is any of your concern, but let me give a little context.

My archive spans five major RAW converters: old ACR, Bibble, AfterShotPro, RawTherapee, darktable.
All of them with have sidecars.
All the metadata I entered in any of them among a quarter million images can be used in any of them.
Yes, sometimes I have to run some conversion scripts to inject keywords from one sidecar into another … but that can be automated.
That’s one side of the story.

The other:
My workflow involves three computers and two NAS drives.
Plus automated backup drives in case of a drive failure.
With sidecars I can just push one folder from one machine to another and know all my metadata will be there.
Edits can - and will - be revised, changed, etc.
But keywords, location and other things remain.
(darktable with their non-IPTC speciality metadata system is throwing some spanners into the works, but it is solvable).

So … reading existing FILENAME.RAW.XMP¹ files is a must in my book.
And while that xmp-lib is already there, well, use it to write your Filmulator section back to it, too.

¹) please ignore Adobe and their stupid FILENAME.XMP standard. It is backwards, not intuitive, produces only problems with JPG+RAW workflows.

Good to know. I’ll keep this in mind.

This sort of problem is why following industry standards is important.

Let me be blunt: you are asking an ISO standard to be overridden. You had better have a really good reason to do so. Chances are very strong the people responsible for the ISO standard know a lot more about metadata and program interoperability than you do. The people responsible are experts in their fields. Are you a recognized metadata expert who is invited to speak at industry presentations, for example? Do you think you know as much as or more than David Riecks, for instance?

If you can produce statements by some of the people responsible for the ISO standard that says “hey we made a mistake, it should be FILENAME.RAW.XMP and not FILENAME.XMP”, then that would be a perfectly good reason to adopt this practice.

Otherwise it’s best to go with what the ISO standard says. That’s what pretty much everyone else does. The only people who don’t are developers who think the XMP standard is “stupid” and assume they know better. In reality they don’t know better, they just think they do. All they end up doing is making life more difficult for everyone else. Files like TIFF and JPEG embed the XMP; files like RAW files use a sidecar.

P.S. it is not just software where deviating from standards happens and causes all kinds of problems. For example, Apple step outside the PTP standard for their iOS devices, which is fine for them and macOS, but extremely annoying otherwise. gPhoto2 doesn’t work properly with modern iOS devices. Windows PTP doesn’t either. What a pain in the rear end. Such arrogance on the part of Apple!

Damon, with all due respect … that part of that standard is a result of someone hacking something together without thinking it through and then just selling it to a committee that did not bother either.

A long time ago, when I was in the alpha-testing team on BibbleLabs and they went from their homemade .bib sidecars to xmp I was one who initially fought them on the filename.raw.xmp idea. But that changed the moment I shot a job in RAW+JPG and was able to edit them side-by-side as I pleased while maintaining complete flexibility. I was happy that they ignored this idiotic piece of the standard. Today those 15 year old xmps open happily in darktable with all the metadata present. Interoperability with any Adobe tools has been a hassle, as always. So yes … the filename.raw decision by hack plus committee was a bad one.

Of course I do not know what the exact story behind filename.xmp is, but the result is clear: use Adobe or die. Other members of this forum have already written thorough explanations, but since your argument trails around “it’s a standard” I have no hope of convincing you otherwise … I’ll just leave this link here:

1 Like

I’m for whatever’s logical, ISO standard or not.

1 Like

Indeed, the common understanding is that metadata shouldn’t be written (embedded) directly within RAW files. But here’s an interesting analysis made by the ExifToolGUI creator saying that it might not be that risky to do it: Writting metadata: To raw or not to raw?

Filmulator keeps track of files in its internal database by hash… if you edit the metadata and move a file and try to teach Filmulator the new location it will not recognize it as the same file.

Perhaps in hindsight I should have figured out a way to hash the image data specifically, but that’s set in stone for now.

Given the maturity the metadata tools have these days, I agree with the general idea.


From a maintenance point of view writing to RAW files is a nightmare. Or even original JPGs out of cam for that matter.

The data ratio between RAW vs XMP is approximately a factor of 100 (JPG) to 5000 (RAWs). So any change you make in your editor will result in any disk operation to take at least 100 times longer than they need to. And that is for measly JPGs.

With average RAW files these days hovering between 20 and 40 MB and XMPs seldom exceeding 20KB we are talking a factor of more than 1000. For every single operation.

Add a keyword to 100 files? Change the whitebalance on 50 files? Insert your contact information into 10000 files? Run backups? Maybe over the internet? Everything will take longer.

You might be able to do it, but it is not a really clever thing to do.

1 Like

FWIW, I think that’s fine. :slight_smile:

I would avoid writing anything to imported files. Those are precious and, in fact, in my workflow, are generally read-only (they are locked by git-annex). Writing to the sidecar is fine, of course, and so is writing to another JPG file, where embeddeding the XMP makes sense. But I would never write to the originals.

1 Like

I know them in the interface as multiple levels of parent / child tags. Just like a tree view.

Imaging adding a tag ‘animals’. Under that a child tag ‘pets’. Under that add child tags with the names of the pets.

Now during import / organize I can click on the name of a pet. And that enables to picture to be found under the name of the pet, but also under generic ‘pets’ and also under generic ‘animals’.

Now, how that tree view is stored in a xmp I have no clue. Xmp looks like xml to me, so the tree hierarchy could (should :)) parent and child tags in my mind…

… But I wouldn’t be surprises as that someone somewhere decided to make a standard about splitting the levels with a separator character :(.

Is there a distinction between that and just having all the tags individually?

Is it a way of making sure the main tag always comes with the hierarchy above?

From what I’ve been able to see of lightroom, it also uses the pipe dileaneated list for hierarchical tags. The left most tag is the top of hierarchy.

<rdf:li>landscape | Yosemite | Half Dome </rdf:li>

My thoughts on photo industry metadata standards and our community:

Maybe - to be sure - we need to clarify what I mean by tags. I mean the literal tags you can assign to aid in searching. I do not mean all the fields that can be written in a xmp block, like title, lens, date, etc…

Doing a search the term ‘tag’ is confusing since it’s also used for the field names (the keys if I see a xmp file as a simple key/value store concept).

So it seems the field ‘subject’ is what is used to store the taglist. And like I suspected, this was once envisioned as a list of subject names, and someone decided to hack a tree hiearchy in it be using a separator.

According to metadata - Is there a well-known way to store a taxonomy hierarchy in XMP, IPTC or EXIF? - Photography Stack Exchange (stack exchange and stack overflow are the world most contributing members of false info so take care checking :)) digiKam has a different way of doing it.

So, two things :

  1. the pipe separator seems to be the most used. Why that way… I don’t know. Be compatible with programs that don’t want to implement a tree view?

  2. It highlights the problem with xmp: you can write tons of stuff in it, and make up field names as you go. A lot of programs make custom field names prefixed with something (like lightroom’s ‘lr:’) which makes the list of field names to check and parse larger and larger and larger… And meanwhile users expect every program to read everything ‘because it is a standard’. It hardly is,it’s just xml.

So, if you write sidecars (with the intention of keeping users happy who want to migrate data in and out) xml is the most widely used. But even using a library like exiv2 or a tool like exiftool requires you to think about which fields to write and read.

About embedding xmp.This is safe IF the raw file is a container format which supports writing xmp. Since almost all raw files are based on tiff or other general containers this is no problem. If it’s a weird format without real xmp support, do not go crazy :).

But… In my mind source files from a camera are never ever to be modified.

And embedding xmp is not useful for the use case of most people wanting sidecar files, so I don’t see the use for now. But that’s just my opinion of course.

We can start using the proper terminology: elements and attributes.

This is actually one of the strengths of xmp/xml. You can store your stuff in a custom namespace, but still access and store it alongside the standard xmp elements. The stuff you explicitly want to share goes into the commonly supported xmp namespace and the stuff that doesn’t need to be shared goes into the custom namespace.