What are the flaws with this solution? It seems super simple to me. The only cost is the mental adjustment of having the basename describe the file content and the extension the format available. Both are actually the norm.
The idea that the metadata, in the form of exif and iptc should match one and only one file makes no sense in publishing or any other dam situation?
By adjusting to the very basic file naming scheme you can suddenly manage your files as their content not their files. A magic gain in abstraction and practicality. Whilst still avoiding databases and speciality tools. Again Geeqie does this completely naturally (grouping sidecar extensions setting). Tag the content not the 3 files. 50% less work with one quick stroke.
So I think we’d all agree that basename.extension.metadataextension is technically a better solution but basename.metadataextension seems to be the “industry standard”.
Now, in my opinion, it is not important why basename.metadataextension was chosen in the past, it’s not that great but what’s done is done and we have to live with it.
Now what is important is what that industry standard brings to the table? Why would we choose to go with the a little bit less flexible implementation. What would we have to give up and what would we gain?
If the gains are greater than the losses then I think we should go for it, if not then we shouldn’t.
I’ve never thought or cared about XMP metadata, it’s just not that well communicated in the community as a very useful thing. I think it’s time I and everyone like me takes another look at that and try to see some potential there.
Now @damonlynch mentioned that by adhering to the industry standard it would gain us the interoperability with other software and with online services.
I won’t pretend to know how that works on Goggle Photos, Flickr or other places and software. But I will say that he brings up a very fair point! Because that’s a big enough reason for many users to choose to make a trade off and go with the “industry standard” way of doing things.
If all of them are using basename.metadataextension then we might be loosing more than we are getting by diverging from the “industry standard”.
I would also like to hear @anon41087856 and @wpferguson opinions on this. They are developing a WordPress plugin that would publish photos from daktable afaik. I’m guessing they’d want to take a look at and evaluate potentially utilizing the xmp files too! In that endeavor I’d probably want to go with the “industry standard” if I ever wanted it to go beyond Darktable. But that’s why I’m mentioning them here to voice their thoughts.
One day maybe we’d like to have a Nextcloud photo extension that also makes use of the XMP metadata. We’d like that to adhere to a wide standard too I guess.
Anyway, I’ve learned two things today. I should care about properly naming my files as it just makes sense, it is nice to have and it’s very easy thanks to RPD.
And I should care about the xmp metadata and should look into how I can use it to my advantage.
My own opinion is that Darktable should definitely support the “industry standard” because the potential pain points pale in comparison to the gains and some users might be willing to make that trade off. It’s not the best possible thing, the best would be for IPTC to change their standard but it is what it is and we have to get some and give some.
There might be an option in the settings to change from let’s say Darktable standard to IPTC standard when creating xmp files. That way everyone would be happy.
I’d like to extend this discussion a bit. I’d like to know in what way and for which purposes are people using the xmp metadata? How do online services make use of those exactly? I only ever thought about those files as just history, but I’d like to know more.
I’d also like to ask people what they think of this thing? https://github.com/darktable-org/darktable/pull/2819
Obviously design is just horrible but personally I’d like to see a powerful metadata editor in it’s own dedicated view space in Darktable. But it must have a much better design and UI than it’s propose in this MR.
It would be a cool way to enrich our images and maybe start thinking about metadata too, not just the photo itself.
The case for technical superiority of filename.ext.xmp has so far been based on xmp as a config file for raw software. The advantage for tagging metadata in the form of exif, xmp and iptc has not been adressed? I have adressed why filname.xmp is advantageous but havent really seen counter arguments by anyone actually using the features. You can always freely add and remove tags from individual jpegs if you wish but then you are no longer in sync and loose some batchability. A fair choice.
Not only that: I DO have different metadata (including tags) for different exts of even the same file. It’s advantageous for me, it works for my usecase, it’s flexible… Why would I not use it?
yes - because “commercial” apps don’t understand filename.ext.xmp.
I haven’t received a reply on why can’t “commercial” apps support filename.ext.xmp? Why can’t they be more robust in terms of .ext support?
And I want to ask - It’s 2nd thread on same topic with same pushback from open source devs and same arguments. Has the topic creator or you created support questions/threads for commercial apps suggesting they could support filename.ext.xmp too? You know, just to be fair?
I think everyone is aware of the spec allowing edits to be stored. However this is a separate issue and much less critical as edits can’t be fully compatible between software.
The choice of using xmp for edits is of much, much less concern for users and more an issue of one less sidecar in the folder and what format the developers want to write their configs in. It has very little impact on users. If config is stored in xmp it’s crucial to follow the spec though to avoid breaking the exif/iptc part.
Is it though? Not even Adobe, the creator of the XMP standard, holds to its own standard for handling all types of metadata. Take a look at what Premiere Pro produces after analyzing video files:
This is still metadata according to Adobe’s own definition: “Metadata is any data that helps describe the content or characteristics of a file.” Apparently the company who designed and pushed for adoption of a standard is okay to store this type of metadata differently? But some people think it is really bad that not every (FOSS) software follows the XMP standard? Aren’t those double standards?
This seems the core of the argument, really. There are multiple levels of metadata. Metadata per file, metadata per group of files, even metadata for multiple groups of files, and so on. These different levels of metadata seem hard to reconcile if you restrict yourself to a single XMP file.
Along with the different organizational levels also come different types of metadata. On a per file basis you may want to store things like creation date, but perhaps also some color label or ranking (for example, if you have multiple renditions of the same image that you label differently), and also things like editing history. Then on a group basis you may want to store things like shooting location, descriptors of the content (‘family pictures’ etc.), things that would result in redundant information if stored on a per file basis. And one level higher you may want to store information about author.
So, how does the XMP standard allow for this kind of control? And how does it rhyme with the idea of using the filename.xmp convention?
In my mind the master child relationship is important and inluences how metadata propagates. For a reasonable workflow you tag the master file, and any sooc children, with all the relevant metadata in one go. Creating the master .xmp file that applies to all files with identical basenames.
All files produced from the master will then have the base metadata included. Those children will naturally have some properties that diverge be it colour space, resolution, changed content due to crop file format specific metadata, notes that help identify it etc. Depending on the changes and workflow a rename (suffix perhaps to maintain relationships) can be made. You can also embed additional metadata in the child file as long as the format allows it. When outputting the file again overwriting it, after tweaks say, you have to redo that extra tagging but that’s inevitable. Allowing tagging only at export would be an interesting feature.
Of course you can maintain any personal metadata system on top of this if you wish but this can’t be expected to propagate or be understood buy the world. xmp and the standardised tagging is there to for that data you want to work externally.
That restriction is only for files with identical basenames in the same folder that haven’t been individually tagged. Not much of a restriction imho.
^ I’m not sure this is was an intentional retorical repost or a mistake as I answered it in the post you replied to.
As a user you can to whatever you like with your tagging and metadata. Override with fine grained per file embedded metadata, run parallell system, have a database, a text file, all metadata in file names etc. This is all possible whilst having well behaved standards compliant software than can interface with the rest of the world.
I’m super confused about this, a joke that this thread is to heated risking to undermine free software? A conspiratorial anti-capitalist worldview? @johnny-bit is expressing similar “rebellious” anti corporate stuff but it’s pretty hard to understand. Perhaps because I doubt anyone else on this forum has as “hard”, long term and “extremist” views on those issues as I do. There are only a few positions below and none to the left of me on the political compass.
You’re trying an anti-corporate BS as a counter argument against “the standard”?
I’m saying it clearly that filename.metaext is inferior and even @Thanatomanic showed that Adobe itself can use filename.ext.metaext. You say it’s “not much of a restriction”… Sure it’s not “that big of a deal”, but clearly allowing filename.ext.metaext is more flexible and robust for ALL standards that relate to file connections.
So, to circle back to anti-corporate BS: I have nothing against corporations (in this discussion), I have nothing against actual standard. I have a problem with interpretation that filename.metaext is “THE STANDARD” and therefore filename.ext.metaext is bad.
Simple: You guys are drumming for filename.metaext to open source devs, have you drummed for filename.ext.metaext to commercial stuff? I think more flexible and robust interpretation of filename is generally better than blind reliance on less flexible and less robust first implementation.
From my angle there’s no significant difference between those to so that’s probably why I didn’t get the joke.
Oh @johnny-bit just read things again please. The text you quote contain no argument about standards what so ever. It referred to you meme with “corporate” appearing dumb.
As for the rest of your post it’s also all over the place.
I think the filename.xmp standart is better than filename.ext.xmp regardless or the standard that should be clear by my extremely numerous posts.
Since the well adopted standard uses filename.xmp “we” should to regardless of my feelings.
We’re not discussing any metadata sidecars besides xmp ones. There are lots of possibitilies for lots of use cases and metadata formats beyond xmp but those are irrelevant to this thread and can use whatever conventions they see fit.
Convincing the entire world of DAM, publishing software and commercial raw developers to use a, in my view worse, solution is a completely different thing to arguing the case that a couple of foss softwares whose developers I’m in contact with should adhere to the standard. It takes a special mind not to see that.
Sorry to interfere. A total ignorant about metadata standards…
After reading every post here, I still have a doubt: as pointed in the first post, metadata is really valuable when publishing images or giving information to the viewer. As a standard de facto, publishing online requires jpegs, and those files embed metadata inside them. Same for tiffs and pngs (AFAIK). So if a FOSS tool correctly saves metadata into the headers of those files, why would I ever need a file.xmp sidecar? Doesn’t Google and others read inside the images?
If so, why a raw development app must ever need to write anything about the raw file into a file.xmp file? All in all, there will be no interchange of developing information between different raw developers, so the only excuse for that file would be metadata. Anything else would have to be recreated/redeveloped in a different software. So where’s the need to interchange that? I mean the information about what you have done with the raw file.
If all the problem is metadata, wouldn’t it be enough having a button in every free raw developer to «export» (save) just the metadata in a file.xmp sidecar? Just in case you need to change to a different raw developer for good? And if you are not going to change app, then leave to the developers create the sidecars they wish/need? Why is that a problem? I just need to upload a single file (e.g. a jpeg) to Flickr, not the sidecar, as the jpeg has all the needed information. Or not?
Because the raw file is the master. Typically multiple files will be exported from this file. At a minimum you’ll have a srgb jpeg file for web use and a wider gamut jpeg or tiff for print and other media. By tagging the master with the base metadata all child files will automatically contain that info. So all things like creator, keywords, as well as most other iptc fields will flow from that file.
If you only tag output files you have to do that every time you export a new one which can be an awful lot of times. This is error prone as well as time consuming. If your output format has dubious metadata support or the recipient demands it you may send the xmp along with the image to ensure all metadata gets shared/uploaded.
Whether a raw development app should have a metadata editor is up to the scope of the software. To work well it does need to read and propagate metadata correctly following the xmp standard. “technical” tags might well have to be written by the raw developer to xmp for it to work but I care less about the technical metadata so I don’t know about this. Its fully conceivable to do all the metadata editing in a separate specialised software but then agan the raw developer has to read an propagate it.
Absolutely, this would work. But it’s better not to have a second file.ext.xmp laying about next to it confusing matters and software.
You think that filename.xmp is better, i (and evidently couple other devs) think that filename.ext.xmp is better.
Still - there would be absolutely no harm for “the standard” to also adopt checking for filename.ext.xmp. Literally. That’s maybe 3 lines more to check for it’s existence…
It takes a special mind IMHO to not see that this is a bit… “Ohhhh big commerical has a std… and FOSS devs are stubborn that the std is a bit limiting… So rather than bothering big commercial that FOSS have solution to limitation imma bother foss devs to criple their solution”. That’s how I see the argument.
Wonderful thing about foss seems to be that one can talk with the devs, argue, file a feature requests/bug reports etc… With commercial software you pay and don’t get that level of sway.
Another wonderful thing about foss is: PRs welcome. So if you don’t like something you can actually fix it yourself!
Ok. I agree that there must be a standard about placing certain information under certain tags.
I agree that after tagging the raw file, any exported image will include those same tags.
Don’t follow this: if I export an image with my preferred raw editor, I expect that any tagging I have already included in that raw will be automatically added to the exported image. I don’t have to do anything. Or do I?
No matter the way the raw developer stores its raw metadata, I figure it will be correctly included into the exported image.
If you mean that if I add new tags, then I would have to re-export all previous exported images, …, well, then read next quote.
If I follow you, in the file.xmp scenario, I don’ have to re-export the (e.g.) jpeg file. And if the recipient demands it, with the updated file.xmp everything is fine…
Well, I read some time ago about iptc vs exif vs xmp, and how are they supported, but I can’t seem to find it right now, so let’s assume this old post explains a bit how convoluted the situation is (and I would love that it was not that way, but…).
So, if what is in that post is true (that is, each program defaults to certain metadata storage), there will be a bunch of programs (including, shockingly, Adobe PhotoShop) that won’t read certain tags in the updated xmp file, if and only if those tags are already included in the jpeg. So to put it clearly: I exported a jpeg including certain tag in its header, then I updated that tag in file.xmp, and finally I sent both. The program may only read the tag from the jpeg header because it takes precedence…
I don’t know, but I guess the standardization problem goes way farther than just including a middle extension or not. E.g.: to be sure a program (any program) reads correctly the metadata, it should be stored in all three storages (iptc, exif and xmp). And that presents a whole bunch of other problems, I think.
As I already said, no matter what the program uses to store its metadata (even in a xmp file including processing parameters), if you exported a file, then sent it somewhere, and after that you had to update the raw metadata, which is asked by somebody, just click that button and you will have your desired file.xmp. Then it has to be seen if the recipient already sees the updated metadata or not (it will depend on the program).
