Free software and photo metadata — a chance to engage with the broader photography industry

I don’t have that habit. Maybe that’s something I should change, to have more descriptive file names.

I store hundreds of thousands of images but I’ve never renamed a raw file in my life. I’ve always relied on the folder structure alone. I’ve never used RPD in my life. After reading RPD documentation now, I think I’m gonna change that and introduce RPD in my workflow.

1 Like

My Nikon D800 allows me to change the first three characters, so files that come from that camera are named “AGA_0724.NEF” etc. Problem solved.

I still don’t understand why adding what’s probably a 3 lines of code to check for both filename.xmp and filename.ext.xmp is a problem.

5 Likes

It’s not a problem for the raw developers. It’s a problem because it breaks a huge machine of institutions software and people. It breaks the meaning of certain relationships. You can’t code your way around that. The standard i set up so that meaning and relationships are conveyed via file names. (which is the purpose of file names to begin with, inode…)

standard is good
much confuse
corporate

5 Likes

Worth also reviewing the discussion from last time this was raised, which includes some good points as to why darktable uses the mechanism it does: Linux applications and their non-standard XMP sidecar naming convention

filememe *

You still havent answered if you actually manage your metadata.

*note the conclusion could also be they all have different content. Which is equally wrong.

Different example:

It’s common for text documents to exist as word processing files and pdfs for sharing and publishing. Would you have a file structure such as

  • dokument1.odf
  • dokument1.pdf

And they are different dokuments? When downloading source code would you expect

  • projectsourcecode.tar
  • projectsourcecode.zip

to have different content?

In Filmulator there’s an option to append a hash to the filename if you are importing a file from the memory card to a directory structure.

Standard ?
Let’s look at at the list of metadata to see that’s a joke, a real mess, every big one adding its own versions, without talking about file naming, of course.

So, instead trying to be fully compliant with that mess, I think it’s sometimes more important to be flexible and to be able to provide the right info at the right place, in particular when one publishes images. That’s where it is essential to control the level information to be disclosed.

1 Like

I do manage my metadata. And I do rename all my files to *captureDate*-*title*.*ext*. I do shoot raw+jpg, and if I feel like developing the raw I replace the camera jpg with the jpg produced from the raw. That’s simple: One picture, one name. I also use face tagging (locally only). Face rectangles do not align between the created jpg and raw in general. They don’t even fully align between the out-of-camera jpg and raw (internal distortion correction). With *name*.*ext*.xmp there’s no problem or ambiguity doing that, the mapping is clear. With *name*.xmp when batch renaming I need to rename raws and jpgs from cameras separately to get different names - that’s just pointless complexity. And I know xmp can be embedded in jpgs. However then you have embedded metadata plus a *name*.xmp file - which one is the applicable one?

A different thought: Are IPTC, xmp, … standards really concerned with filenames? That seems like a secondary afterthought. Those standards go to great length specifying what information is to be stored, and how it is serialized. I’d say that’s their main purpose. Whether to read from a header, *name*.xmp or *name*.*ext*.xmp seems like a minor detail that can easily be handled by sane defaults and configurability (e.g. look in all places and use the one that is present, and if present in multiple places apply a certain order, or try to merge, or whatever).

1 Like

It should be noted that darktable also does use any *name*.xmp that’s present on initial import, in order to load data from xmp files that might have been created from Lightroom. In not subsequently overwriting that file (by using a different naming convention) this file remains available should the user wish to go back to Lightroom again. That seems to me like the nice thing to do.

1 Like

Are you actually in good faith arguing that same basename should have same exact content across different extensions?

here, lemme at ya:

  • file.c
  • file.h
  • file.o

I sure well expect them to be different.

This arguing FOR having .xmp file with just basename instead of full filename is trying to enforce bad idea just for the sake of preservation assumptions that are imo wrong (or the standard was implemented wrong and people go with it because admiting to mistake or correcting the standard is too much to ask and there’s always gonna be people supproting silly ideas… Like in Poland people voting for PiS)

As for storing XMP in media files:

  1. non destructive editor should never touch insides of a file. even just for metadata
  2. jpeg has limit of 64k for header container. my xmp files sometimes exceed twice that.
  3. afaik not every format even allow storing data in its headers/containers so good look with those.

darktable is my non destructive editor + DAM so… yeah? And it allows me to include nice selected tags inside exported files and i can have all the goodies: like duplicates with different purpose, multiple tags, differentiation between filename.ext1 and filename.ext2 is no problem etc… Why would i even try to use inferior solution?

Oh, and I noticed something: in prev discusion there was a mention that technically filename.xmp can differentiate between filename.cr2 and filename.nef in same dir via separate parts inxide xmp files tagged with extension… THAT is probably one of more convolved workarounds for simple problem that I’ve seen this week (and I code in PHP @ work). So… If i want to move the .nef file out do i not take .xmp or do i take .xmp along with data for .cr2 file? what if metadata for .cr2 is considered private/confidential and .nef isn’t? “Well, separate them in different dirs or rename…” Well I say “good day to you” sir, you are not my mum and you won’t tell me what I’m to do with MY files.

“all non visual tools”? shiet, a simple file command can differentiate between those. exiftool & bunch of others too! And still - forcing people to rename their files or assume equal content based on basename without extension is bad.

Oh, that’s nice… there’s also

  • dokument1.odt - a presentation notes
  • dokument1.odp - a presentation
  • dokument1.ods - a spreadheet showing all the nitty gritty data
  • dokument1.zip - archive with all those files…
  • dokument1.zip.txt - a plaintext file containing info how to obtain password for the zip file.

yea, if one enforces people to follow bad standard with huge naming constraints then sure, they’ll use it, but don’t go around saying that “constraints are good for you” while we sure damn well know they aren’t.

4 Likes

If your file and exiftool commands can see which of the nef and cr2 have the same content as the jpeg I really need to switch to a more updated distro.

I think you are misunderstanding what content is. We’re not talking bits but content as in what the data describes. Egg, horse, london etc. its semantics not hashes. A cropped file can have the same content. Alice might also be cropped out but then a rename is apt.

Those files are all “sidecars” to the same conceptual entity named file. You are writing my arguments for me.

There I completely agree with you. If the content differs move (as in /bin/mv) the file. I can see how the the above solution will “work” but its not elegant and most can very easily avoid it.

I can see how for some photographers that there is no meaning or content in the files. They are aesthetic things where the subject doesn’t really matter. I can also see that some never have to dig out files from years back or collate a bunch of files for some funeral, lecture or publication. The metadata is for those who do these things.

Oh come on, you’re being silly on purpose. file.nef, file.jpg and file.cr2 are and should be threated as completely different files with completely different content and no assumption about their corelation should be made.

rename isn’t apt. When I do photos i don’t even care about filenames, only that they are unique in given folder. All I care is what’s inside them. And for me that’s all visual + tags + ratings + data + edit history… all that in darktable :slight_smile:

Mep… fine… all those could as well be illuminants.h, temperature.c, whitebalance.o and content stays same. (just a tiny change in .c file) and it changes NOTHING…

I do metadata for files and my example of IMG_0724.CR2 and IMG_0724.CR3 comes from that too - those files are in same dir for same theme since those contain also files from 2nd shooter… AND HAVE DIFFERENT METADATA.

zoidberg

fortunately darktable has xmp files named .ext.xmp and thus those aren’t standard xmp, those are “eXtreme Megametadata Philes” :stuck_out_tongue:

keep your standard to yourself if you like it so much. the way darktable does it is IMHO better and more flexible.

You’re confusing Exif with IPTC. See Types of Metadata | Photometadata.org

So you abandon the whole concept of file names as bearers of meaning despite the whole origin of file names are as bearers of meaning. You are doing something very ambitious here. Arguing againts semantic file names. That’s bold although some have done that before you. Those that want to completely hide the file system abstraction.

Which proves that you have no use for xmp files. Darktable would in your use case be better off with .dtb files (example). Xmp is explicitly about a standard for metadata to allow the (relatively) smooth transfer of metadata between software, people and organisations. To adopt the sidecar .xmp extension whilst breaking compatibility makes it worse.

dtb files could use xmp structure internally and use the filename.cr2.dtb provide excellent separation of edits between individual files and would stay completely clear of messing with xmp. Why use .xmp when you don’t follow the spec? There is no advantage to it as I can see.

This ties in with discussions above in the thread. Does it make sense to store edits in xmp files or are we better off with software specific sidecars such as .pp3. Here I would lean towards it making sense to separate edits and semantic+camera settings metadata. It seems cleaner to me.

The cost being users silently losing IPTC metadata they’ve taken the time to enter when they transfer images between systems.

1 Like

OK, to sum up: XMP standard has broken handling of different filles having same basename and different extension. Instead of aguamenting standard to support the situation, it’s better to force broken stuff on people because “muh standurd”. K.

I do know the difference between both.
Could you detail a bit more which confusion you talk about ?

Put aside our discussions on XMP. You’re setting yourself up for a whole world of pain because you use duplicate file names. There are excellent reasons why the world has books and training courses on image management. Use unique file names is pretty much lesson #1.

It is fair to assume that the XMP standard assumes a certain baseline level of image management competence.

If you don’t want to manage your images then you’re on your own. No one can help you unless you help yourself.

If you have IMG_0724.CR2 and IMG_0724.CR3 expect a problem with your XMP sidecar being overwritten when you edit the XMP metadata.

Just like if you export IMG_0724.JPG from IMG_0724.CR2 and then a few minutes later export IMG_0724.JPG from IMG_0724.CR3, expect an overwritten jpeg file.

It’s the same principle.

1 Like