darktable 4.8 location of lensfun database for personal use

Morning everyone

I’m a non-developer, so trying to figure this out by trial and error.

I’m using darktable 4.8 on Ubuntu 23.10 for my Fuji GFX50s files. I have the 35-70mm lens and a Venus Laowa 20mm zero-d shift (a fully manual lens without electronic contacts).

Neither of these lenses appear to be supported in darktable’s implementation of lensfun lens corrections.

I would like (eventually) to learn how to calibrate the corrections for these lenses and submit them to the lensfun database for inclusion in the official package. But in the meantime, I want to play around with the install on my machine and figure out how to set up everything correctly.

I vaguely understand that it should be possible to manually create xml files and add them somewhere so that darktable will pick up my custom files and if I use lensfun-update-data, to get the latest database, my custom files will be protected and not be overwritten.

I’ve tried following various bits of documentation online, forum suggestions and so on, but no luck so far creating custom files that get picked up by darktable.

I therefore decided to edit the “official” install xml files, just to see what happened. I found xml databases at ~/.local/share/lensfun/updates but messing with this (or even deleting it entirely) did nothing, likewise another copy at /usr/share/{local/}lensfun. Clearly, these location were not being used by darktable.
Eventually I found a copy at /var/lib/lensfun-updates/ that appeared to be the active database.

I edited mil-fujifilm.xml to add dummy entries for my lenses and had some success in getting them to show up in darktable’s camera and lens lists in the lens correction module. This is a start.

But I want to add the custom xml files in the correct location, rather than hacking the database. My question is where do I store such DIY files in the folder structure and what name do I give to the xml file/s I create?

Thanks in advance

David

Maybe these would help? Unfortunately, I’m not knowledgeable regarding lensfun, my old Nikon SLR lenses are supported out of the box, so I’ve never had to do anything.

https://lensfun.github.io/manual/latest/dbsearch.html
https://lensfun.github.io/manual/latest/lensfun-update-data.html

According to those, your own files should go under ~/.local/share/lensfun, and those will not be overwritten; updates are written to:

  • ~/.local/share/lensfun/updates/version_x (without sudo)
  • /var/lib/lensfun-updates/version_1 (with sudo)

On my system, the commands behave as described in the docs:

kofa@eagle:~$ lensfun-update-data 
[...]
Successfully updated the database in /home/kofa/.local/share/lensfun/updates/version_1.
kofa@eagle:~$ sudo lensfun-update-data 
[...]
Successfully updated the database in /var/lib/lensfun-updates/version_1.
1 Like

Thank you, this is extremely helpful.

I can confirm that when I ran sudo lensfun-update-data and lensfun-update-data, I got the same messages and paths that you did.

I deleted the mil-fujifilm.xml file from /var/lib/lensfun-updates/version_1 and darktable then lost all knowledge of my camera and lenses (proving it was using this file)

I then copied the mil-fujifilm.xml file to the ~/.local/share/lensfun folder that I newly created and darktable regained knowledge of my camera and lenses (proving it picked the file from the “own files” folder. This is great!!

However, when I created a file called dave-lensfun-database.xml and populated it with some test data, darktable did not detect this new profile. I suspect that this means that the xml is extremely sensitive to accurate coding and there is something amiss with my custom xml that is causing the custom file to be ignored.

1 Like

UPDATE:

I created a file called dave-lensfun.xml and placed it in ~/.local/share/lensfun

I added the following code to the file:

<lensdatabase version="1"> 
     <lens>
        <maker>Fujifilm</maker>
        <model>GF35-70mmF4.5-5.6 WR</model>
        <mount>Fujifilm G</mount>
        <cropfactor>0.79</cropfactor>
        <calibration>
            <!-- Taken with Fujifilm GFX100 II -->
            <distortion model="ptlens" focal="55" a="0.0108298" b="-0.0385622" c="0.0226601"/>
        </calibration>
    </lens>
    <lens>
        <maker>Venus</maker>
        <model>Laowa 20mm F/4 Zero-D Shift for GFX</model>
        <mount>Fujifilm G</mount>
        <cropfactor>0.79</cropfactor>
        <calibration>
            <!-- Taken with Fujifilm GFX100 II -->
            <distortion model="ptlens" focal="20" a="0.0108298" b="-0.0385622" c="0.0226601"/>
        </calibration>
    </lens>
    <lens>
        <maker>Venus</maker>
        <model>Laowa 20mm F/4 Zero-D Shift for Sony FF</model>
        <mount>Sony E</mount>
        <cropfactor>1</cropfactor>
        <calibration>            
            <distortion model="ptlens" focal="20" a="0.0108298" b="-0.0385622" c="0.0226601"/>
        </calibration>
    </lens>
</lensdatabase>

Everything is working correctly: the two lenses’ profiles are correctly displayed and selectable in the lens correction drop down menus and the distortion correction is properly applied.

Yay, this is what I was aiming for!

The next step is to run the calibration images and generate the proper correction values to replace the dummy ones I have at the moment.

I was broadly working on the correct lines but must have been doing something minor wrong that broke the process. Your tips got me back on track and it’s now working has hoped.

Many thanks!

Dave

2 Likes

Here is another link:

1 Like

How to take pictures for calibration is described here (section “Taking pictures”) and partly also here and here. The images can be uploaded from this website. Please store the images separately for each lens in folders (distortion, tca, vignetting) and provide the file names of the images with the focal length and aperture. The calibration can then be carried out by the members of the lensfun project. The other option is to do the calibration yourself and make the data and some raw pictures for testing available to the lensfun project.

Here you will find some hints:
calibration with lens_calibrate.py and taking pictures
calibration and taking pictures
How anti-distortion images should look like
video 1 calibration with Hugin
video 2 calibration with Hugin
lensfuns coverage
lensfuns database
calibration with calibrate.py