I’m the principal contributor to Exiv2 which is a C++ Metadata Library. It’s used by GIMP, DigiKam, Darktable and other applications. If anybody’s interested in this, I’ll be happy to discuss the library.
4 Likes
@clanmills Hi Robin, we met at LGM this weekend (you gave me your calling card at least) but for whatever reason I didn’t talk with you (maybe because I didn’t want to chime into the talk of you and a dt guy at Saturday ;-)) However, in RawTherapee we currently use ExifTool, but I really would like to check exiv2 too. So, yes, I would like to get some informations about using exiv2 from you and also to discuss it here!
Ingo
1 Like
Thank you for all your work! I’d imagine an underlying lib is a rather thankless job. I enjoy your work all the time in the form of correct metadata.
1 Like
Hi. I think you were in a gang of folks who arrived about 09:45. Such a fun and busy day, we didn’t manage to talk. I must get round to looking at RawTherapee (and many other things). Exiv2 is a C++ library, so you don’t need Perl. I don’t know exiftool well enough to really compare it with Exiv2. Phil has done a great job with exiftool.
To get started with Exiv2, you can download and build the code (for Mac, Linux, Windows). http://www.exiv2.org
We have a build server, so you can download the daily build for your preferred platform: http://rmillsmm:8080/userContent/builds/
Thanks. I really like the Exiv2 library. I didn’t write very much of it. That honour belongs to Andreas, Brad, Gilles and Volker. I’ve dealt mostly with the build for the last 8 years. However, I’m been keeping the project alive for the last 3 years. I retired in 2014 after 14 tough years in Silicon Valley, so I have time to spend on this. It’s more-or-less an unpaid full-time job, however I find it stimulating and keeps my hard-won C++ skills alive and sharp.
Yes, exactly, we arrived about that time 
Anyway, I’ll try to get started with Exiv2 following your links next days.
Thanks a lot and really sorry about not having a talk with you 
Ingo
Keep up the great work Robin! Exiv2 is a critical piece of infrastructure for anyone working in open source photography.
In the other applications field, probably the biggest is Shotwell which is the default image management application installed by many. Shotwell uses gexiv2 to talk to exiv2 about metadata.
I would also mention the exiv2 cli amongst the list of applications.
Thanks Robin!
Thanks, Alan. I believe I’m on course for code-complete on Exiv2 v0.26 in early June. I intend to “stick around” for v0.27 in 2017. After that, I’m so sure.
I wonder where we’ll find my replacement. I’d like to leave the project when I’m confident that one or more C++ engineers will maintain libexiv2, the exiv2 cli program, and the test/build infrastructure.
I’ve sometimes wondered what metadata library the popular proprietary software tools used. I had assumed they pretty much all use exiv2 simply because it’s such a huge and complex project – it would be insane for the proprietary developers to reinvent the wheel – and because it’s written in C++, which I assume is what most proprietary photo tools are written in. I had also assumed that the revenue from the commercial license fees funded costs associated with maintaining exiv2. Is that not the case?
Hi Robin! We also briefly met in London (I’m the PhotoFlow developer) and it has been really a great pleasure!
PhotoFlow entirely relies on Exiv2 for the metadata handling, so I’m greatly indebted for your hard work!
By the way, I would like to take advantage of this discussion to ask a technical question: does the library provide an “official” way to dump the metadata into a memory buffer (text or binary) that is suitable to be embedded into VIPS and/or GIMP metadata holders? In GIMP I have found a custom piece of code that chains the metadata tags into a string buffer using GExiv2, but I would prefer to avoid duplicating this piece of code if a cleaner solution exists…
Thanks for all!
It’s my mission with Exiv2 to get Apple/Google/Adobe/Microsoft to discontinue using their home made code and adopt Exiv2. The reason is to improve metadata reliability and application interoperability. I was an Adobe Senior Computer Scientist in San Jose, California. I am happily retired in England. I didn’t work on metadata at Adobe.
One of the reasons for the success of the internet is Apache. The world only needs a single reliable cross-platform web-server that implements standards. Microsoft’s web-server IIS is fine, however it is far from plug-compatible with Apache. One system of solid web plumbing is better.
I’d like to see one system of solid metadata plumbing. Exiv2.
Apple/Google/Adobe/Microsoft should save their energy. Instead of writing everything themselves, those companies should participate in Exiv2. Customers would be happier. They would save money, Exiv2 would have more resources. Everybody wins. This is a common development model. It has been very successfully adopted by GCC, Clang, webkit and other ‘heavy hitters’. A heavy hitter is a huge library such as webkit with millions of lines of code.
Exiv2 is 100k lines of code. There are many open-source libraries which are used extensively by ‘the big guys’. Examples: zlib (27k), expat (17k), curl (123k), openssl (350k).
The politics of how to involve ‘the big guys’ in Exiv2 is a mystery to me. I write code. I don’t do politics (or more accurately, I do politics badly!). Perhaps Exiv2 needs an Evangelist!
1 Like
Dr Ferrero: I was really impressed by your talk in London last weekend. I not a photographer, although I’d like to be! The team-work in the open-source community is very impressive. You’ve build on the shoulders of VIPS and produced something amazing. Open source rocks. I’m privileged to be a part of this movement.
This is an interesting question about writing the metadata. The library has an internal model to represent metadata in different formats such Exif, IPTC, XMP, ICC. Metadata is serialised into different file formats such as JPG, TIFF and PNG according to the format specification.
I don’t know anything about how VIPS and GIMP deal with metadata. If they are reading/writing standard file formats such as TIFF, I don’t know that you need to write metadata to memory.
Exiv2 does have a ‘neutral’ way to encode metadata without graphics into a .exv. This format is actually a JPG with no graphics! Exiv2 has an API to write the metadata to file and we support in-memory files. So we can write all the metadata to an in-memory file which you can store/read as you please. The data is binary encoded.
You can examine the internal structure of common file formats with exiv2(.exe) -pS path or exiv2(.exe) -R path. -pS = print Structure (in v0.25) -pR = print Recursively (on trunk for v0.26). The option -pR recursively prints sub-structures of the file.
I added a sample application in v0.25 exiv2json which serialises metadata as JSON (and therefore ascii/human readable). At the moment, I don’t have a decoder to read json formatted metadata.
I didn’t answer your question about the commercial license and revenue. I know almost nothing about this. Andreas doesn’t ask very much for the license, however it is totally his business. I have no financial involvement in the project. In fact, I purchased the MacMini which runs the build server. It sits on my desk at home and I use it for other purposes which have nothing to do with Exiv2.
I don’t know how many commercial licenses exist. I suspect it’s quite small. When folks ask questions on the forum, or report issues, the type of license is of no importance to me. The reason is because I think the commercial license is simply a mechanism for a company to ship a product using the library and they are free of the obligations of GPLv2. It feels to me as a legal matter and has nothing really to do with the Exiv2 project. So whether somebody is deploying Exiv2 using the open source model, or commercial, supporting the developer is the same.
If Andreas were selling commercial support contracts I would want to be paid. However he’s simply selling, at modest cost, a mechanism to ship without being constrained GPLv2.
The costs of running the project are small. We have an AWS node running exiv2.org on which we have an SVN repository and Redmine Forum. I assume that Andreas covers these costs. I’ve never discussed it with him. For my own part, in addition to purchasing the MacMini, I pay the annual charge for the buildserver’s dynamic domain: exiv2.dyndns.org
Thanks for the explanations Robin. VIPS and GIMP read common file formats and their metadata, but then this metadata needs to be attached and propagated through the processing pipeline in the form of a single memory buffer, hence my question.
The solution of writing to an in-memory Jpeg with no graphics seems to be quite reasonable, and should work at least in the VIPS case. For GIMP, I’ll need to experiment… in this case, my main problem is how to pass the metadata from the PhotoFlow plug-in to the GIMP image structures in a buffered way that is understandable by both programs.
Is there an example of writing an in-memory JPEG?
Thanks for all!
Perhaps we need a 3-way (or 4-way) conversation (on Skype?) with somebody from GIMP/VIPS because each of us understands about 75% of what’s needed, however none of us understand 100%.
XMP is “Extensible” Metadata. Perhaps that’s where you should write your raw-processing data. So your data shouldn’t be hidden in a place only known by GIMP, it should be in the XMP stored in the image file.
I talked to Pascal from Darktable about having generic raw-processing metadata in the image. This would enable different raw-processor/editors to work together. Pascal didn’t believe this was possible because every raw-processor has different filters/parameters. Because of my strong background in PostScript, I like device independence. You should express the user’s intent and allow the host renderer to determine how to apply it’s built-in magic to obtain a result.
Writing a JPEG (and therefore a .exv) to memory is illustrated in jpgimage.cpp/JpegBase::writeMetadata() where we open a BasicIo::AutoPtr and call doWriteMetadata(). To write the metadata to memory, we need to call JpegBase::doWriteMetadata(BasicIo& outIo) on an instance of BasicIo::MemIo. I’ll develop the code and publish it on the Exiv2 Forum.
Hi Robin! Sorry for answering late, but I had to catch-up with daily work after some short vacations…
In fact, the problem I’m facing is not the storage of the metadata into the output files, but the internal memory storage of the same data inside the processing pipeline, in a way that is understandable both by photoflow and GIMP.
An example: I load and develop a RAW file, and then later on in the processing pipeline I need to apply lens distortion corrections. For that I need to know the camera model, lens model, focal length, etc…
However, at this point of the pipeline the original RAW data is not accessible anymore, so I need to “propagate” the metadata together with the pixel data through the pipeline.
At the moment I am doing this by simply attaching an instance of the Exiv2 object to the pixel data, by this does not work when it comes to the point of transferring the metadata from PhF to GIMP…
As far as I understand, GIMP internally stores the metadata in the form of an XML-formatted text buffer. The relevant pieces of code that save and read back the XML buffer can be found here and here.
The GIMP code uses GExiv2 instead of Exiv2, but I guess one could achieve the same XML conversion directly with the Exiv2 API… would this be something that could be considered to be included in the Exiv2 library? If not, I will most likely stick to my current implementation and use the GIMP functions when outputting the metadata from the photoflow plug-in to GIMP itself…
Thanks!
Gosh, I don’t know the answer. In fact, I’m not sure I know the question! Maybe we are overloading the term “metadata”.
In my case, “metadata” is persistent Exif, IPTC, XMP and ICC data in image files. However, perhaps you’re using the term “metadata” to mean all of mine PLUS data concerning raw processing which you wish to share with GIMP. It feels to me that storing your data in XMP is appropriate and would be visible to any/all applications.
However, it’s quite possible that I have not understood your situation and if you have a working method (using the GIMP/XML mechanism), perhaps that is the correct solution. For sure, if you are transferring information “in memory” between you and GIMP and that information should not be persistent in the image file, then it’s unlikely that Exiv2 is involved.
I have been working on our discussion to “serialise metadata to memory” and I think I’ve encountered issues/deficiencies with our MemIo class. So this subject is “work in progress” and I’ll add a topic to our forum in the next few days about this.
1 Like
No, we are in fact talking about almost the same thing. In my case, “metadata” refers to Exif, IPTC, XMP and ICC data associated to the pixel buffers that are processed in the pipeline. The concept is basically the same, except that the metadata is not permanently stored in a file on disk but kept into a memory buffer that goes together with the pixel data.
Anyhow, in the specific case of transferring this metadata between GIMP and my plug-in, it turned out that passing around GExiv2 objects was the right solution.
Nevertheless I think that a mechanism to translate the metadata handled by Exiv2 into a text buffer that can be saved and read back by Exiv2 objects would be a useful addition to the API, and alternative to saving the data into a Jpeg file without pixels…
The RAW processing parameters are yet another topic, for which as far as I know there no standardisation yet (and I’m not sure there will be one at some point…).
Maybe it would be a good idea to have a closer look at the GIMP code I’ve been linking to in my previous post, as it seems to provide already 99% of what is needed to serialise and deserialise the metadata into an XML buffer…
Yes. I think I have understood this. The BasicIo::MemIo class in exiv2 is intended to handle images in memory. We can serialise the metadata to a .exv file stored in a memory block. I’ve been working on that again today and added samples/toexv.cpp which does the job and will show you what to do. I have not finished yet as I have discovered bugs in MemIo. In the next few days, I’ll open a issue about those bugs, fix them, and update samples/toexv.cpp to show you what to do. I will also include a work around for existing versions of exiv2.
I only glanced over the GIMP/XML code. I’ll have a more careful read at that over the weekend.
Have a nice (long?) weekend. Speak next week.
Unfortunately I’ll be away from home and working during the whole weekend
That’s life… but that does no prevent me to wish you a nice weekend! 
1 Like