Another question about lensfun and darktable… I see there are a few questions on lensfun and darktable and I have looked both here in this forum and www for many hours without much luck.
I dont get to use darktable that much as the Adobe Lightroom Mobile or Snapseed apps are so handy for mobile DNGs but for the real DSLR raw files I am looking at using darktable more.
My question is How do you add a new xml file to the lensfun folder and have darktable see this new xml in linux mint?
Simply creating a new xml file from a couple of lcp files with lensfun-converter-lcp and leaving this xml in the default output folder should be enough according to a video tutorial on you tube
however it isnt. Manually moving the xml to the default lensfun folder doesn’t work either. Updating the data base with lensfun-update-data gives a lot of new xmls in a version 1 and version 2 folders and these new xmls can be seen by Darktable. Adding a new xml to the updated folder does not work. Editing an existing xml and adding the new information inside a file in any of the default folders for lensfun also doesnt solve the problem. Darktabel does not seem to be able to see a new manually copy pasted xml. Peculiar because there are people who seem to have been able to do it without problems on their window machines or even in Ubuntu using a snap Darktable.
While trying all sorts of things I now have lensfun folders every where so to speak…
I cant help but think that I am missing the obvious but I am at a complete loss. Darktable can only see what is in the lensfun folders if it has been put there via lensfuns own app. Any manual file or editing of an existing file is ignored. Is it possible that I have to somehow update the data base timestamp or does Darktable keeps its own database somewhere else which needs to be updated manually?
The standard location for user-generated lensfun profiles is ~/.local/share/lensfun. The video you mention appears to put them there. If you have an issue with the video, perhaps you might contact the author.
Note that while you can also put them into /usr/share/lensfun/version_x/ (where x is 1 or 2, corresponding to the version of the XML file) this is not recommended as they can be overwritten by your package manager or other automated tools.
This usually means that the XML is invalid. (It has information that is unsupported for the version you’re using, or you inserted it into the wrong place in the file, etc.) If you put the file(s) into ~/.local/share/lensfun and they were not detected, it’s likely that the XML itself has an issue.
After I updated Lensfun from Terminal the database darktable use is now in /var/lib/lensfun-updates and in version1. I use Ubuntu 20.04.
I tried to convert from LCP files but never got it to work. It never worked for me to add acm in slr-canon.xml either. Don’t know what went wrong.
Add a fake lens in slr-canon.xml for every database you have and name them 1, 2, 3… Then you may find which database that is in use.
I’m struggling myself. This is what I found:
Converting Adobe lens correction files (*.lcp) with the lensfun-converter-lcp tool will produce lensfun profiles version 2 xml files. All Adobe lens files use the Adobe correction model, short “acm”, not the “ptlens” model from Hugin. Only lensfun version 2 xml may use this “acm” model. To check start darkTable or RawTherapee from the command line: they will spit out errors about the wrong model “acm”, even with darkTable 3.8.0.
Why does it work in the video? Well, there is a hint that the video producer uses Arch Linux. Under Arch Linux darkTable is compile with version 3.95.0 of lensfun, not with 3.2.0 like the other distributions. There are API changes in lensfun 3.95.0 which break the build of an unmodified darkTable. I don’t know how Arch developers fixed this.
But I wonder: the release notes of lensfun say, that even lensfun 3.2.0 should support version 2 xml correction files. If there is a folder “version_1” I would suspect that they should be read from a “version_2” folder, but I did not try yet. And darktable 3.8.0 uses lensfun 3.2.0.
I had no success putting model “acm” xmls unter ~/.local/share/lensfun/mil-panasonic.xml
But even then there are no Panasonic lcp files because these corrections are embedded in every Panasonic RW2 file, but this is a totally different problem.
What we would need would be a API compatible new lensfun release and a darktable 3.8.1 release using this so version 2 corrections would work.
A bit of an update on the Adobe DNG converter - Lensfun LCP converter and Database Version 2 saga.
I think its not just me who thinks that finding lensfun correction profiles is not that “fun”.
Running Linux Mint 20.3 (Ubuntu 20.04) I use ART (Rawtherapee - Appimage) and Darktable latest (when it doesn’t crash on me).
To my surprise the ART appimage 1.14 had no problem finding the lens correction files for a mobile phone Huawei P30 Pro in its database but the system installed Darktable couldn’t do the same.
I thought they used the same database but apparently not. I am guessing the appimage is build with its own DB.
Checking the Lensfun folder I could only find Version_1 and in the misc.xml file there was indeed no Huawei profiles.
Finding the lensfun github I saw that there was a new data base just release a few days ago so I downloaded the ZIP. This data base is version_2 if I understand correctly.
Now the Lensfun github did show me that the Huawei P30 Pro was in there. But as I only had lensfun database version_1 I would have to get a data base version 2 and since I do not know how to do that manually or even how to build it from the github I took a shortcut (I know this is all linux but there should be a bit for windows/macs coming up). I found an older .deb for liblensfun and installed that. It got me the database version_2 on the machine and probably other system related things needed under the hub. Next thing was to update the Database to get the Huawei profiles. The lensfun update tool did nothing except look at database version_1 but (and here is where it might be useful for windows and macs who I believe doent’t have any update tool) I downloaded the latest database from the github, unzipped it and manually copy pasted the profiles I needed (The files with the profiles not the profiles) into the folder with database version_2. Now my system installed Darktable could see the Huawei profiles no problem (I am thinking seeing windows and macs also have an issue updating this should work for them as well).
In short (form) and with links
Download latest Lensfun database
Code - Zip
The (linux) update program lensfun-update-data is (you need to install liblensfun first) supposed to update all database folders (so both version_1 and _2) but I could only get it to look at _1 which is not updated. https://lensfun.github.io/manual/latest/lensfun-update-data.html
Instead (and this could also be useful for windows users as mentioned by kmilos How change to the deveopment version of Lensfun) I unzipped the Lensfun.Zip and copied the files from the data folder (my Huawei is in the misc.xml) to the lensfun folder version_2 (as the updated database files are version_2). Said yes to overwrite (needed root for that).
Started up Darktable and look and behold it now found a lens correction profile for the Huawei P30 Pro.
So not a question on how to do what but perhaps helpful for others getting caught up in the fun of lensfun.
I am having issues with lens correction data for appimage versions of darktable and ansel.
And the video mentioned above is not playable any more.
Does someone know which files have to be placed to ~/.local/share/lensfun?
The correction data is not missing for my lenses, so that’s not the problem. But they are not included in the appimages.