A Metadata Rant, and some AI insight

Processing
1

So, I’ve been using exiv2 in rawproc for a while now, with various snafus and inconveniences. The straw that broke the camel’s back was 7sec save times to my network storage, 50% owing to writing the un-metafiled image first, then reading it back with exiv2 to add the metadata. So, I decided that’s enough, the v27-28 API change and its reticence to handle PNGs at all just added up and I tore exiv2 out of rawproc. I now save ‘essential’ tags with the libjpeg/tiff/png libraries, works fine. So, notice to the one or two of you using rawproc, there’s that in master, and making its way to 1.5

I also had an AI experience that expanded my insight on the whole thing. In the original file handling of rawproc I’d never figured out the “extra” EXIF tags, e.g. FNumber, ExposureTime, ISOSpeedRatings, and FocalLength, so in the re-implementation I decided to get that right. My original code still didn’t work, so I first decided to get some AI -cogitated routines to at least study. Geminii through google gave me a similar implementation that also didn’t work, and Copilot, whereI got good code for my CAD endeavors told me that the capability to do those tags did not exist in libtiff, first tried to get me to use libexif, then gave me code to build the IFD by-hand. Geesh…

So, I finally just went to the libtiff documentation, where it described exactly how to lay out the code. Spent less time trans-coding that than I did arguing with Copilot. And, in the process I learned a little bit about TIFF organization.

Back to metadata, I got to thinking again about my approach, which is to rely on the raw file to retain all metadata associated with the capture, and populate renditions with only what they need. To that end, I really didn’t need to put fstop/shutterspeed/ISO/focallength in renditions, except to communicate here with other photographers. The real need in renditions is 1) image width and height, because software needs that to lay out the image bytestream, and 2) embedded color profile, so color-managed destinations can do their thing. To work with a rendition , is there anything else?

Correspondingly for raw files, here’s my list: 1) image width/height, both raw and visible area, 2) black subtract value, although I think that could be computed, 3) ‘as-shot’ white balance, for a starting point, 4) mosaic pattern, and, 5) lens distortion and correction data for camera/lens systems that require it (a new one). Right now, to get #5 for rawproc I’d have to engineer an exiftool-based collection mechanism, as that’s the only “library” I know that recognizes and decodes those tags. Really, I already do that for a display tool, wouldn’t be hard to make the data structure available to other code. Libraw makes an effort to supply all the metadata needed for raw processing, may engage them about including it.

Anyway, just sitting here thinking of stuff, always a dangerous occupation…

6 Likes
2

I faced similar challenges when making my image viewer, in the end I resigned myself to just call exiftool through a (or multiple) subprocess and then parse its json output, or just dump it directly into the database, which makes it searchable through JSONB sqlite commands. I really wish EXIF had some standardized fields fields for stuff like Rating, Tags, etc.

1 Like
3

Isn’t that what XMP is for?

2 Likes
4

In all my dealings with data, from telemetry from orbital vehicles to environmental data loggers I’ve never seen such an hodge-podge of formats and encodings as in digital photographs. Early on, I decided not to carry the full raw entourage to my renditions, except for fnumber, shutter speed, ISO, and lens focal length.

Oh, and the rawproc toolchain that made the rendition, crammed into the EXIF ImageDescription tag in direct defiance of the definition. I only took on exiv2 to see if its encoding made my metadata more readable by those web decoders at various photo sites, but it really didn’t, so I pulled it out and went back to the image library routines.

The stuff Adobe adds to an image doesn’t seem to be of use in re-processing the source to the rendition, still need a freaking sidecar for that. Oh, sidecars, maybe another rant…

1 Like
5

:laughing:

1 Like
6

My wife ordered three 75-keurig-pod boxes of Starbucks esspresso roast. I should probably lay off those… :crazy_face:

2 Likes
7

Okay, sidecars…

Never really liked the concept, didn’t like that I had to Mary-had-a-little-lamb the file with the associated rendition to keep things organized. Seemed to make more sense to store the processing recipe in the rendition itself. Really, that was one of the impetuses behind rawproc, my hack raw processor.

When rawproc saves a rendition, it saves the toolchain that made it in the EXIF ImageDescription tag. Keeps me from using that tag for other information, but don’t really want to document things about my images. So, if I go to open a JPEG, TIFF, or PNG that rawproc created, rawproc will see the toolchain and ask if you want to open the original file and re-apply processing, or open the rendition itself. The former is usually what I want, so rawproc opens the original file, usually the camera raw, and applies the toolchain from the rendtiion. I now have a starting point to do other stuff, like change the size (my original renditions are usually 800x600 proof JPEGs), maybe crop it a different way, or do more non-default processing. I can re-save to the same image, which gets the updated toolchain, or to another file, which gets that toolchain. So, every rendition in my picture collection has its processing contained therein, no pesky sidecar to keep track of.

I like this mechanism so much, it keeps me from considering tools I might like better, like vkdt. The only thing I don’t have I might like is an AgX tone curve, may snarf some code and see how to shoehorn that into rawproc.

But, never a sidecar, never.

2 Likes
8

interesting take, thanks for sharing. indeed sounds appealing to simply use tiff tags and libjpeg (you do it even without libtiff, nice!). is your jpegexif.cpp some derivative of libexif? code looks familiar but i can’t remember why. i may want to use your 1k lines of code there to replace exiftool…

proof jpeg are a sidecar too, no? embedding processing in exported images is simple, darktable has done this since forever. vkdt processing is plain text which would be a better fit for textual descriptions (but i don’t embed it).

i don’t store proofs (if you don’t count thumbnails in ~/.cache/), if i want to see a picture i’d process the raw again.

1 Like
9

Heh, I’m exploring incorporating an exiftool read in order to get Nikon lens distortion/vignetting correction information. Something to be said for exiftool, you get the latest as well as the most-est metadata.

Just went through jpegexif.cpp history, I can’t for the life of me remember what I did. A little worried, I thought I was doing well crediting things like stackoverflow snarfs, of which I think this has a few. I do remember the frustration trying to get metadata to work, spent more time on that than almost all the algorithms combined.

In my workflow, every rendition carries its processing toolchain. The beauty of that comes in the opening of a rendition; if it contains a toolchain, rawproc offers to instead open the source file and apply the toolchain. So, when I get to my computer after a shoot, I batch process the raws to proof JPEGs. Then, if any need more work (highlight weighted metering guarantees this) I’ll drag the proof to the rawproc icon and it’ll ask for the source-process option, I’ll select that and get the processed raw as a starting point. Another thing I inadvertently built in was the batch tool’s ability to use the current toolchain; if more images need the same change, I delete their proofs and run the batch tool, which will re-create those proofs with the new processing, cool beans for ETTR adjustments.

If someone wants a full-sized rendition (get that a lot in the railcar restoration thing), I just re-open the proof as described above, delete the resize tool, and save to a different filename. ta-da.

My wife just uses the proof renditions for Facebook and other dissemination, which speaks volumes to the over-and-above we nerds do for our images… :crazy_face:

I should do a video, makes more sense when you see it at work.

1 Like
10

Yep, it’s also pretty fast if you leave an instance running or execute it for 500 or so images at once (when reading). The biggest overhead is the Perl startup time

1 Like
11

Sidecars are a great idea for metadata storage in a file system. Particularly where there is some value in keeping the original file untouched as metadata is updated. I don’t like the way Adobe and darktable use the xmp sidecar for storing software edits though. I’ts much better imho to have software specific sidecars as RT does it. This leaves the xmp file for “real” metadata about the content of the files. Such metadata is much more persistent for me than the edits which in practice I often redo or tweak for new publication. The edit data is also a bit fat and verbose.

Managing the files are a non issue to be as the raw files stay where they are and correctly configured most output formats embed the data. This is also a super useful thing about the xmp standard that all files with the same basename inherits the xmp sidecar and it’s metadata.

1 Like
12

To my way of thinking, the original file metadata is about the moment of capture, so I wouldn’t want to change any of that. Then I make one or more renditions, and they have their own metadata, some of it copied from the original (e.g., fnumber, etc.) and some a manifest of its creation (in my world, toolchain).

There are other needs. I’m working with a fellow who’s managing historical engineering data, e.g., images of drawings, blueprints. He has a number of them every year that time-out copyright-wise, go to public domain, and he wants those to go to archive.org “self-contained”, that is, all relevant information recorded in the image metadata. He keeps spreadsheets of all the artifacts, and I’m writing a program that inserts some of that information into relevant tags of the PD JPEGs and TIFFs. For his tasks the spreadsheets form a sort of ‘sidecar’, but the artifacts can’t be accompanied by such when they go off-campus.

Sometimes you feel like a nut, sometimes you don’t…

1 Like