Call for testing: RawTherapee metadata handling with Exiv2 (includes CR3 support)

It’s currently in the beta testing phase, so there will only be updates in response to bugs and usability concerns. You can track the latest changes in the pull request mentioned in the original post. I will also share any significant updates here.

Hello,

Everything works fine in the Exif and IPTC data in the metadata tab of the editor display all important information, and editable fields such as EXIF copyright can be edited without any problem.
With DNG(PENTAX K-30), and NEF(Nikon D7100)
I edited an exif tag and it is saved

grep Benoit IMGP0372.DNG*   
IMGP0372.DNG.pp3:Artist=Benoit
IMGP0372.DNG.xmp: <rdf:li>Benoit</rdf:li>

I just compiled after an update of the source in the branch : Nothing to report

% git branch 
  dev
* metadata-exiv2

cmake ../RawTherapee -DCMAKE_BUILD_TYPE="release" -DCACHE_NAME_SUFFIX="5-dev" -DPROC_TARGET_NUMBER="2" -DOPTION_OMP="ON"

About->Version :

Version: 5.9-232-g294c6167a
Branch: metadata-exiv2
Commit: 294c6167a
Commit date: 2023-03-26
Compiler: cc 10.2.1
Processor: x86_64
System: Linux
Bit depth: 64 bits
Gtkmm: V3.24.2
Lensfun: V0.3.2.0
Build type: release
Build flags: -std=c++11 -march=native -Werror=unused-label -Werror=delete-incomplete -fno-math-errno -Wno-attributes -Wall -Wuninitialized -Wcast-qual -Wno-deprecated-declarations -Wno-unused-result -Wunused-macros -fopenmp -Werror=unknown-pragmas -O3 -DNDEBUG -ftree-vectorize
Link flags: -march=native
OpenMP support: ON
MMAP support: ON
Build OS: Linux 5.10.0-21-amd64 x86_64
Build date: Sun, 02 Apr 2023 17:43:20 +0000 UTC
Build epoch: 1680457400

Hello,
with the support of Exiv2 lib my guess is that the amount of supported cameras and lenses will be increasing.
It is already hard in lens correction to find the correct camera and lens, the menus are pretty large. For stuff not found by lensfun, the situation will get even worse.
It would be great to be able to filter the provided profiles to the cameras and lenses, which are in use. No automatic filter, but something configurable (e.g. filter for “Sony” or “Olympus”).

1 Like

I agree. While the scrolling menus are workable, the sheer number of entries has gotten to a point where (without filtering) a pop up dialog box with a scrolling combo list would be easier from a UI point of view. Maybe a dialog with an incremental-match search field. For example, as you type:

S - Sigma, Sony, Soligor
So - Sony, Soligor
Son - Sony

… etc.

Of course I have no idea how much dev effort that represents, though.

The incremental-match is the generic solution, but requires mouse + keyboard interaction.
Since the cameras and lenses of each user are limited and static, I would prefer a fixed filter entry in the config file (like setting my favorite tools tab by vi, or like in the preferences → file browser tab you can configure the allowed file formats in a graphical way).

We’re throwing developer time and effort around :slight_smile: but as long as we’re “wishing”, a GUI with a both full and favorites lists would meet both wishes. One small, user-editable list of favorite lenses and another full list of all supported lenses.

I usually work on images from only my lenses, but not always. Between friends, Play Raw and other resources I virtually “touch” various lenses from time to time and having to edit a file (then presumably restart the editor) would be clumsy IMO. Plus, I’m already using a mouse / keyboard while editing anyway.

But what’s already implemented does work. However it will reach a scaling threshold at some point…

I agree that some sort of dialogue/filter UI solution would be nice. Meanwhile, what I’ve been doing is creating a few partial profiles for my manual lenses. Then, it’s just a matter of selecting the right partial profile file when I open a raw file taken with a particular lens.

Thanks for testing @benoit!

@Jens1 @lphilpot I have also noticed it can be hard to find certain cameras and lenses when the lists are long. GTK probably supports list filtering, so adding a search feature could be as easy as flipping a switch. As noted by @MarcosC, partial profiles can be used as a substitute for a favorites list. Combining them with dynamic profiles allows for automatic selection if the lens and camera details are present in the metadata. If you only need lens corrections for your own cameras and lenses, you can create a copy of the Lensfun database, stip the unneeded profiles, and tell RawTherapee to use the modified database

1 Like

I see (by reading the exif with exiftool in command line) that the modification of the exifs is not written in the raw file but in the associated file (pp3* or xmp)
I suppose that it is wanted

Yes, that is the expected behavior. Modifying raw files, even if it’s just the metadata, is risky. RawTherapee will never change or delete your images without being explicit about it.

Thanks for this hint, this would work for me.

I have a Sony ILCE-7M2, lenses are “Sony FE 28-70mm F3.5-5.6 OSS” and “TAMRON 70-300mm F4.5-6.3 Di”. Images are in .JPG and .ARW.
In the RT Metadata Filter, the camera is detected correctly.
For the .JPG images, the lenses are showing up as “FE 28-70mm F3.3-5.6 OSS” and “E 70-300 F4.5-6.3 A047”. That’s correct.
The same images in .ARW format, the lenses all show up as “Manual lens”. That’s not correct.
Looking into the .ARW image data with exiftool show the correct lenses.

Probably off topic:
I have an Olympus Tough TG-4 with a fixed mounted lens.
The camera is detected correctly, but the lens is reported as “Unknown”, but should be “Olympus Tough TG-4 & compatibles”.
Exiftool reports no lens information from the image data, but since the lens is a fixed mounted combination with the camera: is there some hard coded mapping existing in RT or is this an issue with the Olypus generated metadata and not related to RT?

Additional information:
The .ARW files have an .xpm sidecar, these don’t contain the lens information.
The .JPG files might (unintentionally) refer to the same sidecar, but does not seem to have a problem.
For the future I might to move to .ARW.xmp, which is supported by digiKam as well.

Sample ARWs would be nice :wink:

Sorry, I screwed up the upload and had no access to my PC the last days.
Here are the images.
The problem does not occur with RT 5.9 and not with the development Appimage.
The existence of xpm sidecars does not change the behavior


DSC01714.ARW (23.8 MB)

DSC01713.ARW (23.9 MB)
.

How do you deal with ambiguous sitecar files?
My images have the naming conventions .ARW, .JPG and .xmp. The .xmp file is only related to the .ARW image. The digikam setting is:

  • Write Metadata into File
  • Don’t write into Raw Files (this enforces generation of .xmp for ARW files)
  • Name of sitecar is compatible with other programs (this causes ambiguity for the .JPG file)

I can change the .xmp to .ARW.xmp, but would like to know whether RT provides some magic for this.

Is that a statement of intent or a description of behaviour? The xmp file is supposed to provide metadata for all files with the same base name. If that’s not your intention you should consider a different file structure (sooc jpegs in sub folder?)

It’s a description of current situation with my images. Since RT as wall as DigiKam both are supporting .xmp and .ARW.xmp, I will switch to .ARW.xmp file structure to avoid the ambiguity.
Thanks, Jens

Update:

  • Fix for lens incorrectly showing up as “Manual lens” for Sony ILCE and NEX cameras.
  • Support for Exiv2 v0.28.0.
2 Likes

It works nice even with phone camera, only “complain” I have is that Im quite positive there is noise/denoise map, but its not used. Unsure what it needs to make that used. Also its most likely for phones only part?

2x2 binning would be also great, but thats asking too much I guess. Altho I think RT is actually great for raw files from phones (DNGs).