My day job is closed for two weeks, so I’m dedicating all of my new free time to improving Filmulator.
My goal for this period is to get Lensfun working. Figuring out the API is one matter, but the first thing I did was to open up some space in the UI for the user to select the camera and lens they’re using.
Lensfun is a bit challenging. I finally made my implementation “full”, with a choice of interpolators. In fact, I was able to reuse the interpolators I’d already included for resizing. Studying lenstool was a big help.
Database maintenance is also “fun”, so to speak. I ended up writing my own update library using libcurl and libarchive, so I could point it all at the directory of my choosing, and include the interface in the rawproc menus. I respected the version_x convention the library maintainers established, as the ABI difference between 0.3.2 and nascent 4.0.0 crosses version_1 and version_2, and there are distro packages out there at both versions.
Finally, lens nomenclature is a real pain. I think a lot of the misses folk bring to the forum have to do with aperture, where f/4.3 and F4.3 are different enough to bollox the match, and there are lensid files out there that use both in the same file version. I’m probably going to incorporate exiv2 just to anchor that somewhere that the lensfun database can also respect…
FWIW you can always inspect my code for one person’s implementation;
It’s Nikon’s doing only because they apparently until recently have not included lens nomenclature in their NEF metadata. This Lens ID scheme is a hack to cross-reference home-made nomenclature to an amalgam of metadata tags. Of note is Nikon’s recent cameras communicate a lens nomenclature, in the EXIF group no less…
Here’s the Lens* metadata from a D7000:
glenn@caliente:~/Photography/rawproc$ exiftool -n -G -*Lens* DSG_3111.NEF
[MakerNotes] Lens Type : 14
[MakerNotes] Lens : 18 200 3.5 5.6
[MakerNotes] Lens Data Version : 0204
[MakerNotes] Lens ID Number : 139
[MakerNotes] Lens F Stops : 5.33333333333333
[Composite] Lens ID : 8B 40 2D 80 2C 3C FD 0E
[Composite] Lens Spec : 18 200 3.5 5.6 14
Here’s the same from a Z6:
glenn@caliente:~/Photography/rawproc$ exiftool -n -G -*Lens* DSZ_4168.NEF
[MakerNotes] Lens Type : 0
[MakerNotes] Lens : 24 70 4 4
[MakerNotes] Lens F Stops : 5
[MakerNotes] Lens Data Version : 0800
[MakerNotes] Lens ID : 1
[EXIF] Lens Info : 24 70 4 4
[EXIF] Lens Make : NIKON
[EXIF] Lens Model : NIKKOR Z 24-70mm f/4 S
[EXIF] Lens Serial Number : 20078247
[Composite] Lens Spec : 24 70 4 4 0
EXIF:LensModel - yay.
The only reliable way I can think of to handle this is 1) use exiv2 as a first-cut source, and 2) have a robust manual lookup of the Lensfun database to let the user pick one. For rawproc, I wrote a list control that lets one filter the list based on a text box, like, as you type, the list in real-time cuts the members to those that match. Works pretty well to find a lens.
After spending the entire morning thinking about it, I decided that I’m going to display what information I can without having to maintain a lookup table.
For example, on older Canons I’ll display the lens name (they don’t have focal ranges or max apertures) and on Nikons I’ll display focal ranges and max aperture (they don’t have names), and attempt to auto-match… but when the user actually processes images, I’ll store the info that shouldn’t change image-to-image in a database together with what camera and lens model pairs they selected from lensfun.
That way the user gets perfect matching accuracy once they’ve gone through all of their lenses, and for future imports it’ll remember which of the three fixes (CA, distortion, vignetting) they prefer to apply for each lens.
I’ve been struggling to identify Nikon lenses, even using the makernote lensID from LibRaw and this page from exiftool.
First off, the focal length and apertures as they’re recorded are weird. 50mm lenses are recorded as 80? 45mm lenses are recorded as 76? 35mm lenses are recorded as 68? I just made a huge spreadsheet of all the stuff and it’s a mess. It’s gonna end up having to be a lookup table, but at least it won’t change often.
Secondly, LibRaw’s lensID borks on a lot of Nikon files (not all though!) so I can’t rely on it; I reported a bug. Maybe I’ll have to do this in exiv2 and see if it’s better.
There are some f/2.4, f/2.7, f/3.8, and f/7.2 lenses lurking in the list, so I would have a few more cases.
I don’t want to encode for every lens individually, I just want to represent the lens specs correctly to help the user pick it from the list the first time.
The values above I evaluated from my old MF Nikon lenses, for which I can dial in the max aperture into my Nikon D700 and just read them out from metadata. No chance for Aptertures like 3.8 or so to evaluate exactly
Great news, LibRaw fixed the bug with LensID already. Now I don’t have to fuss around with grabbing a dozen different exif fields and concatenating them myself.
I’ve just learned that Nikon isn’t the only brand to hide the lenses behind lens IDs… and other brands are even more inscrutable, with just a number to associate with lenses. Ugh.
Turns out Exiv2’s command-line application can interpret these makernotes and tell you straight up what the lens is, which is nice!
But I can’t figure out if that’s something the API is capable of at all, which stinks.
I don’t wanna have to maintain a list of lenses myself… we need more data sharing of this sort of thing.
@ggbutcher when you say you use exiv2 as a primary source, are you doing it via the API or via CLI?
I’m not doing it yet; still thinking through how to transition out of my home-grown metadata logic. The code today uses whatever Libraw provides; for Nikon lenses I have a copy of the Rottmerhusen nikonlensid.h. For my three cameras and lenses, works fine.
Moving toward a version 1.0 others might use, I’m going to use the exiv2 API.
With regard to lens identification, I’m pondering the following: 1) look for a match from the Libraw data. If that fails; 2) look for a match in the exiv2-extracted metadata. If that fails; 3) prompt the user to select a lens from the Lensfun database. if #3, I’ll add the lensfun nomenclature to the tool parameters for that image so the user doesn’t have to relook it up ever time they open the source image to re-apply processing.
It’s a bit of a mess, when the camera doesn’t provide definitive lens nomenclature.
Step 2 is the sticking point, because Exiv2 gives you wonderful things like 4 252 0 for Pentax PEFs. At least it’s possible to decode the Nikon data into focal length range and apertures…
If I am going to have to maintain a lens ID decoding list for the brands that don’t have anything useful from libraw (even if copied from RawTherapee or Exiv2) then I’d rather share it a la librtprocess.
Maybe just a single header file with a function for each camera brand that returns a lens name when given the ID? I could call it lensjoy, because it goes with lensfun.
I feel your pain, but it’s really a question of, “where’s the best place to do that?”. My vote would be in the metadata library.
One question I’d like to explore is the state of maintenance with respect to the file Robert Rotterhusen started for Nikkor and 3rd-party F mount lenses. To all my digging, it doesn’t appear he’s maintaining it anymore as of sometime in 2018; if anyone here knows better, please post…
With that, though, there’s a second question, would the lens ID scheme he uses to divine a lens nomenclature from the available metadata extend to Pentax and other brands/mounts? I’m guessing not, based on what you reported and what Phil Harvey does in exiftool:
exiv2 the application has the translations for the tags, but for some reason exiv2 the library does not (or if it does, I just haven’t figured it out). That’s the problem.
Ouch, sorry… hmmm, I just went back to my experimentation with the exiftool C++ interface, and found the same thing.
Still, do you want to take on maintaining lens lookup tables? I’m going to look at the exiv2 code that does it, and see if there’s already exposed somewhere in the classes to compel that. If not, I’d rather ask the exiv2 developers to expose something we could use.
