It’s an ISO standard.
You keep saying this but all it means to me is “go buy the standard for 158 Swiss Francs” and even then I don’t even know if everything Lightroom does uses the standard schemas (above there’s dc:subject and lr:hierarchicalsSubject) so it might be a waste of money.
It means that tools like Exiv2 and ExifTool already understand XMP.
It means most of the hard work of figuring this stuff out has already been done by people who have deep experience with the topic at hand.
It means that there will inevitably be compromises affecting the purity of the XMP standard to account for how metadata was handled pre-XMP days. That’s life.
Carvac means that XMP can contain a ton of (arbitrary) data. What do you expect to work from a XMP file? Just tags? Date / import stuff?
I know of people who expect the edits from Lightroom to show up perfectly in another program because they are in a XMP.
Carvac: think about why you code. Do you get joy out of other people using your software? Then listen to ideas they have :). Do you use it for your own usage or workflow, do what makes you happy
you need to find a balance in this. I’m not going to say which way to go, but I find it helps a lot to know what the reason is to do it :).
Now…What I like about 'a database is that it k ows stuff from my pictures, even if they are not on the system.
Having one collection in online backup with a local copy on a NAS means that I don’t have all my files on all my laptops. I use multiple machines to ingest,multiple to edit.Depends in where I am at the moment :).
Having a database of everything, means I can search the database (with small thumbnails) to find files that are ‘in storage’, not on the system.
Having stuff (only) in sidecars makes it more difficult to search, since xmps need to be found and read. And if not everything is on my system,I can’t search sidecars that are online in storage.
So for speedier searches,programs often need some sort of database. Writing sidecars then feels somehow redundant. They always feel like clutter to me.
They help with transferring info from program to program… Yes. But that’s something I seldom do,so I don’t want them around.
Being able to write them helps to not be ‘locked in’ to a single program,so that is a big plus for sidecars.
Acdsee has a thing where they write the xmp section back into the files, so they are always with the file (and copying the file copies the metadata you added like tags). Sounds nice, but changing the source files seems wrong to me (the fact that they are hashed on my sotrage means they are not meant to change a single bit :)).
About ‘film rolls’ (or collections, or imports)…
I learned from Acdsee that hierarchical tags are good for everything. It doesn’t have a concept or albums,groups or collections. Just the folders how I ordered them.
But add a tag 'film rolls, and add a tag under that. Assign it to images (during import for example) and there you have it. Nothing is stopping you from adding images to more than one filmroll this way. And also you can still add tags for things like people, locations, etc…
So…Maybe keep the database and build it how you see fit. But add an option somewhere to export xmp files next to source files. As a single action to do when someone wants to transfer info to other software.
And maybe import xmp the same way (with an option to delete the xmp after import 
).
As for functionality, I think hierarchical tags and ratings are the basics.An exif block (or the same fields as basic exif copies) is also expected I think (gps, make,model,dates, title, author. . To name a few.
And I think people will expect it to use it as a backup or transfer to other computer kind of thing, so serializing the filmulator settings to it is probably also what people expect.
Personally, I’m glad my folder organizing is still enough for me… But the time will come I need more :s.
2 Likes
Hi,
As a kinda “tag obsessional” who has spent too much time trying to figure out how to correctly import/export my photo collections in various applications (digiKam, F-Spot, Shotwell, darktable, etc.) over the years , I totally agree with damonlynch: xmp compatibility is the way to go for features and interoperability.
Here’s the obvious, including apps supporting xmp (with more or less good implementation): Extensible Metadata Platform - Wikipedia
Of course, with FOSS, it’s up to the developers to decide on how they want to spend their time, what features they want to include in their app., etc., but IMHO, relying on a “homegrown” solution would not be a wise choice on the long run (and in the interest of users who usually don’t appreciate any app lock down when they want to use more than one tool to deal with their collection or when they want to move it in another app).
My 2 cents!
1 Like
I definitely understand that people will have existing tag collections, and I definitely need to support that.
I can probably help you unfurl the XMP stuff, should you need help.
Thanks.
First off I need to understand what exactly hierarchical tags are.
Are they actually a tree structure with parents/children, or is it just a flat series of tags separated by pipes?
Depends on what program you’re using. In darktable, its a list of tags separated by a pipe.
See for example:
[XMP:XMP-digiKam] Tags List : Species/Human##Species/Human/Position/Left##Site/Timbavati##Individual/NA##Species/Human/Number/1##Species/Human/Gender/Female
======== made_via_exiftool.JPG
[XMP:XMP-Panthera] Hierarchical Subject : Site|Timbavati, Species|Human, Individual|NA, Species|Human|Gender|Female, Species|Human|Number|1, Species|Human|Position|Left
[XMP:XMP-Panthera] Subject : Timbavati, Human, NA, 1, Female, Left
[XMP:XMP-dc] Subject : Timbavati##Human##NA##1##Female##Left
[XMP:XMP-lr] Hierarchical Subject : Site|Timbavati##Species|Human##Individual|NA##Species|Human|Gender|Female##Species|Human|Number|1##Species|Human|Position|Left
And exiv2 can handle these myriad schemas in xmp files?
Or do I have to manually parse any of these many schemas?
It can read and write them.
This is true even if one were to hypothetically never use another photographic program on their desktop computer. If you upload your tagged photos to Flickr, for example, Flickr automatically reads the XMP tags and incorporates them on the photo’s page, meaning they show up in keyword searches etc. That’s the power of an industry standard.
1 Like
That’s embedded XMP though, not sidecar XMP.
I can get behind that, though, as I tag my manual lens models on Flickr.
If you’re going to go to all the work of supporting embedded XMP, then sidecar XMP is trivial.
Sidecars exist because writing metadata to reverse engineered files like RAW files is foolish and dangerous. JPEG is an ISO standard so no worries there — embed the XMP.
Sidecar XMP for RAW becomes embedded XMP in the exported JPEG 
Looking for sidecars was one of the first things I did when testing out your nice software.
Without them it is impossible for me to use it.
Not that is any of your concern, but let me give a little context.
My archive spans five major RAW converters: old ACR, Bibble, AfterShotPro, RawTherapee, darktable.
All of them with have sidecars.
All the metadata I entered in any of them among a quarter million images can be used in any of them.
Yes, sometimes I have to run some conversion scripts to inject keywords from one sidecar into another … but that can be automated.
That’s one side of the story.
The other:
My workflow involves three computers and two NAS drives.
Plus automated backup drives in case of a drive failure.
With sidecars I can just push one folder from one machine to another and know all my metadata will be there.
Edits can - and will - be revised, changed, etc.
But keywords, location and other things remain.
(darktable with their non-IPTC speciality metadata system is throwing some spanners into the works, but it is solvable).
So … reading existing FILENAME.RAW.XMP¹ files is a must in my book.
And while that xmp-lib is already there, well, use it to write your Filmulator section back to it, too.
¹) please ignore Adobe and their stupid FILENAME.XMP standard. It is backwards, not intuitive, produces only problems with JPG+RAW workflows.
Good to know. I’ll keep this in mind.
This sort of problem is why following industry standards is important.
Let me be blunt: you are asking an ISO standard to be overridden. You had better have a really good reason to do so. Chances are very strong the people responsible for the ISO standard know a lot more about metadata and program interoperability than you do. The people responsible are experts in their fields. Are you a recognized metadata expert who is invited to speak at industry presentations, for example? Do you think you know as much as or more than David Riecks, for instance?
If you can produce statements by some of the people responsible for the ISO standard that says “hey we made a mistake, it should be FILENAME.RAW.XMP and not FILENAME.XMP”, then that would be a perfectly good reason to adopt this practice.
Otherwise it’s best to go with what the ISO standard says. That’s what pretty much everyone else does. The only people who don’t are developers who think the XMP standard is “stupid” and assume they know better. In reality they don’t know better, they just think they do. All they end up doing is making life more difficult for everyone else. Files like TIFF and JPEG embed the XMP; files like RAW files use a sidecar.
P.S. it is not just software where deviating from standards happens and causes all kinds of problems. For example, Apple step outside the PTP standard for their iOS devices, which is fine for them and macOS, but extremely annoying otherwise. gPhoto2 doesn’t work properly with modern iOS devices. Windows PTP doesn’t either. What a pain in the rear end. Such arrogance on the part of Apple!
Damon, with all due respect … that part of that standard is a result of someone hacking something together without thinking it through and then just selling it to a committee that did not bother either.
A long time ago, when I was in the alpha-testing team on BibbleLabs and they went from their homemade .bib sidecars to xmp I was one who initially fought them on the filename.raw.xmp idea. But that changed the moment I shot a job in RAW+JPG and was able to edit them side-by-side as I pleased while maintaining complete flexibility. I was happy that they ignored this idiotic piece of the standard. Today those 15 year old xmps open happily in darktable with all the metadata present. Interoperability with any Adobe tools has been a hassle, as always. So yes … the filename.raw decision by hack plus committee was a bad one.
Of course I do not know what the exact story behind filename.xmp is, but the result is clear: use Adobe or die. Other members of this forum have already written thorough explanations, but since your argument trails around “it’s a standard” I have no hope of convincing you otherwise … I’ll just leave this link here:
1 Like
I’m for whatever’s logical, ISO standard or not.
1 Like