lensfun & darktable is somehow crude ! why no one is appalled by that?

[C_M]

Introduction:
So after recognizing that my lens (Tamron B070) was not showing in the darktable (4.0.0) lens correction, i obviously checked back with my local lensfun (0.3.2) . And sure enough, it was missing in lenstool. after updating ( database and compiling and installing lib and tool ) lenstool would properly find the lens.
However darktable still was not showing the lens.

resolution
After having a look around it seems that you have to update the version_1 of lensfun for the use with darktable (~/.local/share/lensfun/updates/version_1/)
Version 1 is not really in the the official git repo data folder though (which holds only version 2 files)
An ominous script (/usr/local/bin/lensfun-update-data) will download a tar-file from an “unrelated” (insecure, http!) place and decompress the required files for version 1. (which differ only slightly)

in my particular case the PYTHONPATH was missing so the script could not run properly (ModuleNotFoundError)
(export PYTHONPATH="$PYTHONPATH:/usr/local/lib/python3.9/site-packages/")

questions

• I wonder, as the location for the lensdata can be in userspace why does darktable not bother to try to update the lensfun data ? if necessary by utilizing lensfun-update-data. (and yea in my case that would not have succeeded, yet a meaningfull error could have been produced)
• what is the reasoning for having (at least) 2 similar versions of the lens correction data without a conversion tool, but only an ominous sourcing ?

PS:
if a google search has brought you here, then please have a look at:

<Solved>Lense Database not showing Camera+Lense combo

I Install DT from the Master branch like echo ‘deb Index of /repositories/graphics:/darktable:/master/xUbuntu_20.04 /’ | sudo tee /etc/apt/sources.list.d/graphics:darktable:master.list curl -fsSL https://download.opensuse.org/repositories/graphics:darktable:master/xUbuntu_20.04/Release.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/graphics_darktable_master.gpg > /dev/null sudo apt update sudo apt install darktable I have Sony A6300 and the lens is Sigma 30mm DC DN f1.4 The lens is …

[rvietor]

@C_M

I wonder, as the location for the lensdata can be in userspace why does darktable not bother to try to update the lensfun data ? if necessary by utilizing lensfun-update-data. (and yea in my case that would not have succeeded, yet a meaningfull error could have been produced)

I think you answered that one yourself, at least for lensfun-update-data:

@C_M

An ominous script (/usr/local/bin/lensfun-update-data) will download a tar-file from an “unrelated” (insecure, http!) place and decompress the required files for version 1.

Doing that manually means the user evaluates the risk. Having something like that done automatically by any old program is a serious security risk…

And what else would you want dt to do? If the lens isn’t in the database, dt does not have the correction parameters. And just adding a lens description sounds a bit useless…

[paperdigits (Mica)]

@C_M

what is the reasoning for having (at least) 2 similar versions of the lens correction data without a conversion tool, but only an ominous sourcing

There is a conversion tool in the github repo. You just didn’t find it. It converts v2 to v1.

Instead of being appalled, find some time to help improve this project.

[ggbutcher (Glenn Butcher)]

@C_M

what is the reasoning for having (at least) 2 similar versions of the lens correction data without a conversion tool, but only an ominous sourcing ?

Lensfun has had to maintain a version_1 for backward compatibility. To do this, they regularly update the version_2 files in github, then quite regularly make a version_1 from it (using the tool @paperdigits referenced) and post both versions to the “ominous source”. Single point of modification is just good engineering.

All that said, Lensfun is kinda stuck in a version limbo. 0.3.2 is the last stable and reliable version; 0.3.95 has problems that caused me to abandon it for the master branch (not for dt, some other software), which looks to have fixed the problems. 0.3.2 uses version_1 data, and that seems to be what most distros are shipping. I’m not a lensfun developer, but it might be they just need someone to put the time and attention into packaging and declaring 4.0, but such a person isn’t available… welcome to open source software!

Oh, and welcome to the forum.

[paperdigits (Mica)]

@ggbutcher

0.3.2 is the last stable and reliable version

0.3.3 is out @ggbutcher time to update!!

[RawConvert (Andrew)]

@C_M , you have my sympathy, when I tried to understand Lensfun some time back, I found it impenetrable. The info above is the clearest I’ve seen about the two versions. thanks @ggbutcher and others!

How about you take up @paperdigits 's idea and explain how to do it for DT? I wrote this - How to build sigmoid and dt - some time back, to explain to non-tech people like me how to build DT. Simple steps, little prior knowledge assumed. If you covered Lensfun, that would be great. Eg pros and cons of installing versus building; how to build; how to use the conversion thing mentioned above, etc. I suggest do it for Ubuntu since I believe this is the most widely used linux flavour.

[gaaned92]

Yes but very old code, about the same as 0.3.2 ( 7 years old)see 0.3.3 maintenance release · Discussion #1653 · lensfun/lensfun · GitHub
It was issued because the Linux packages of applications using lensfun were distibuted without the packagers caring to update the database. So 0.3.3 contains an updated DB version_1.

@RawConvert I understood that what prevents DT to use the master branch is a correction of a bug in Lensfun that breaks backward compatibility.

[paperdigits (Mica)]

Yes, I am aware. there are a few bug fixes in there too.

[ggbutcher (Glenn Butcher)]

Hmmm, nothing about it on the website. Guess I have to watch on github…

[paperdigits (Mica)]

Yes, on github.

[C_M]

@ggbutcher

n_1 for backward compatibility. To do this, they regularly update the version_2 files in github, then quite regularly make a version_1 from it (using the tool @paperdigits referenced) and post both versions to the “ominous source”. Single point of

1.a i didn’t find the tool because it isn’t used ( that’s partly (a+b) the thing i was complaining about )
1.b the version is neither mentioned in the readme nor in dependencies

2.a no, making artifacts and not loading them from githubs artifact store but (still some ominous non-https) place isn’t good engineering.
2.b if v1&v2 is part of the “intended” release, then make should include the conversion

anyway maybe darktable isn’t fully responsible in that regard but in many others of being crude. however it is still the best tool i know of to organize an image collection (digikam is even worse imo)

PS:
just found that lensfun/docs/make-release-0.3.x.txt at master · lensfun/lensfun · GitHub

@rvietor it is user-space ! no security risks beyond the common … and yes dt could ask if that’s what the user wants before simply acting. I thought that goes without saying .
also no need of adding an useless lens description (which i never suggested. )

[paperdigits (Mica)]

@C_M

1.a i didn’t find the tool because it isn’t used ( that’s partly (a+b) the thing i was complaining about )

If you build a version of lensfun that uses version 1 profiles, you’ll get the version 1 profiles included. You can then run the update script to fetch the absolute newest profiles.

@C_M

2.a no, making artifacts and not loading them from githubs artifact store but (still some ominous non-https) place isn’t good engineering

That wasn’t what was said at all.

@C_M

anyway maybe darktable isn’t fully responsible in that regard

darktable isn’t responsible at all. You are. Its your machine.

@C_M

but in many others of being crude

Being insulting isn’t going to get you anywhere. People put their free time into these projects. If you think you can help improve them, then you should put in your free time too.

@C_M

just found that lensfun/docs/make-release-0.3.x.txt at master · lensfun/lensfun · GitHub

So its all there you just didn’t look hard enough.

@C_M

and yes dt could ask if that’s what the user wants before simply acting. I thought that goes without saying .

I mean this is a really great conclusion to this post… Thanks for your complaints… I guess?

[C_M]

tl;dr:

• again pointing out that there is no conversion done
• it’s just bad user experience
• i am done with this thread

---

@paperdigits

@C_M

1.a i didn’t find the tool because it isn’t used ( that’s partly (a+b) the thing i was complaining about )

If you build a version of lensfun that uses version 1 profiles, you’ll get the version 1 profiles included. You can then run the update script to fetch the absolute newest profiles.

I guess you mean if I would have checked out tag 0.3.3, then version 1 would have been there and the update tool would have again done what it did before …

2 problems :

1. which tags are version 1 ?! the versioning issue is missing in the readme
2. we still have the update tool in that picture

@paperdigits

@C_M

2.a no, making artifacts and not loading them from githubs artifact store but (still some ominous non-https) place isn’t good engineering

That wasn’t what was said at all.

1. it was what I was saying!
2. yea having a conversion tool and converting from one primary source is obviously good engineering. I never challenged that. In fact I didn’t even knew there was a conversion tool.
   Its not what i was complaining about.

My point was that the tool that was meant to update did not bother to gather the one primary source and convert, but rather utilize an artifact from a different place over running a conversion.
And that’s crude at least.

@paperdigits

@C_M

anyway maybe darktable isn’t fully responsible in that regard

darktable isn’t responsible at all. You are. Its your machine.

darktable is using a library it knows it may be outdated and even suggests to the user in a tiny tool tip to run that tool … guess its on me then

@paperdigits

@C_M

but in many others of being crude

Being insulting isn’t going to get you anywhere. People put their free time into these projects. If you think you can help improve them, then you should put in your free time too.

i’m not insulting if i say its crude. maybe you don’t understand the word as I do. what I mean by that is that it is not easily understandable why things happen the way they do, and yes I know “organically grown” software does often do things like that . It is crude just as windows is crude.

maybe you didn’t realize it. but I did put my free time into this by reporting on an UX issue with the software.

@paperdigits

@C_M

just found that lensfun/docs/make-release-0.3.x.txt at master · lensfun/lensfun · GitHub

So its all there you just didn’t look hard enough.

what I meant was look at that crude manual deployment.

However I still don’t know what I should have been looking for.
I looked at the readme and no it doesn’t state anything about the version issues there.
Yea, i could have compiled and stepped through the code and find the usage of the v1, or as i did looking how it would behave if the update was run.
but no code for conversion is actually invoked, except by the user that’s building the artifact.

@paperdigits

@C_M

and yes dt could ask if that’s what the user wants before simply acting. I thought that goes without saying .

I mean this is a really great conclusion to this post… Thanks for your complaints… I guess?

who is “insulting” now?
Even your first reply wasn’t very nice and in my opinion you conveyed that not just in writing, but also in omitting the actual link to the conversion tool.
It is not just an inconvenience to me but also to all the others that may find this post, looking for an answer. (but i guess this is not stackoverflow or something?)

Anyway, your replies have shown me that I REALLY do not want to communicate with you. ( take it as the SNR is too low). So please take that as my last reply to you no matter how tempting responding to you ever may be.

for what it’s worth, hopefully my UX report will be helpful to some of the devs in improving the UX.

---

yes in my opinion the thread is done. i am the problem for not understanding the mysterious ways of dt … its ok.
i already spent too much time on that argument anyway.

PS:
any devs out there, try to look at the docs and try to imagine how a new user/dev would penetrate the source/documentation. often it is not the stupidity of the newbie but rather the enigmatic ways of the docs/code .
Simply ask yourself whom do you write the software for. should the code be hard to read because it was hard to write?
good code explains itself through its unit tests. documentation never is good enough

[paperdigits (Mica)]

@C_M

I guess you mean if I would have checked out tag 0.3.3

Yes I absolutely mean the latest released version of the software. Not sure why you’d start anywhere else.

@C_M

which tags are version 1 ?! the versioning issue is missing in the readme

If you build the library, you don’t need to worry about it, the version is taken care of for you.

@C_M

we still have the update tool in that picture

I still don’t understand why that is an issue for the end user. Run the tool > Get the newest lens profiles > go back to editing. Maybe you don’t love the way that it fetches a zip file from the website of the guy who’s been maintaining the project for years, but well… it works.

@C_M

darktable is using a library it knows it may be outdated and even suggests to the user in a tiny tool tip to run that tool

darktable absolutely has no knowledge if the lensfun update tool has been run or not. It suggest it to you to try and help you and point you in the right direction.

@C_M

guess its on me then

Yes. it is.

@C_M

i’m not insulting if i say its crude. maybe you don’t understand the word as I do.

It isn’t clear you understand the social context of being insulting. You, the person being insulting, does not decide if what you said is insulting or not. But I digress.

@C_M

I looked at the readme and no it doesn’t state anything about the version issues there.
Yea, i could have compiled and stepped through the code and find the usage of the v1, or as i did looking how it would behave if the update was run.
but no code for conversion is actually invoked, except by the user that’s building the artifact.

You’re way off in the weeds. You take issue with the way things are build, despite the fact that they work for a lot of people. So you can 1. use the tool, do what is required to updated, and get back to editing or 2 take the time to try and improve things via code or 3 write some long and pointless diatribe on a forum on the internet, or 4. not use the tool at all.

@C_M

maybe you didn’t realize it. but I did put my free time into this by reporting on an UX issue with the software.

This isn’t the correct place to make suggestion for improvement. That place is here: Issues · lensfun/lensfun · GitHub

But without adjusting your tone, I don’t think anyone over there will take you seriously.

@C_M

but also in omitting the actual link to the conversion tool.

I don’t owe you anything.

@C_M

It is not just an inconvenience to me but also to all the others that may find this post, looking for an answer. (but i guess this is not stackoverflow or something?)

Oh no I’ve inconvenienced you, someone who showed up and was rude and insulting from their first post onward?

@C_M

for what it’s worth, hopefully my UX report will be helpful to some of the devs in improving the UX.

Again, this is not the appropriate place for that kind of feedback.

@C_M

i am the problem for not understanding the mysterious ways of dt

Again, not darktable. Lensfun. They’re separate projects.

@C_M

PS:
any devs out there, try to look at the docs and try to imagine how a new user/dev would penetrate the source/documentation. often it is not the stupidity of the newbie but rather the enigmatic ways of the docs/code .
Simply ask yourself whom do you write the software for. should the code be hard to read because it was hard to write?
good code explains itself through its unit tests. documentation never is good enough

Super smug-- this attitude will get you nowhere. Glad you’re not responding any further.

[jorismak]

@C_M

Simply ask yourself whom do you write the software for

This answer is almost always ‘for the dev himself/herself’. Almost never for the end users. Even if this might not be true , as a user it’s best to assume this. It means that the way you approach questions and requests is different , and more likely to be inviting to spent time on.

[rvietor]

@C_M

good code explains itself through its unit tests. documentation never is good enough

Code is for programmes/developers, documentation is for end users. Those two groups might overlap, of course…

[hannoschwalm (Jens-Hanno Schwalm)]

@jorismak

This answer is almost always ‘for the dev himself/herself’. Almost never for the end users. Even if this might not be true , as a user it’s best to assume this.

OK, generally agreed. From my understanding all devs write code with other users fully in mind. It’s more like: alls devs write code based on personal decision. Just have look what gets done in dt… There is so much just because of user interest and one of the devs took on.

[jorismak]

Yeah thats a better way of saying it.

Even if the goal is user adaption , in the end work gets done on what A dev wants to work on . And that can be something else to what other people find the most important:).

It’s a hobby for the devs (even if there is sponsor money or something ). That should always be in te back of the mind when asking why things aren’t done in a different way .

About lens fun … is there even an alternative? None come to my mind so quickly …

[hannoschwalm (Jens-Hanno Schwalm)]

@jorismak

About lens fun … is there even an alternative? None come to my mind so quickly

1. There is an old pr using makernote data, there seems to be no progress there
2. As i have a camera producing dng files having lens data as specified in dng i sort of strted working on that. (There is so much more to do being more important

There seem to be only a few people actively working on lensfun. I don’t speak C++ fluently btw

[RawConvert (Andrew)]

@jorismak

About lens fun … is there even an alternative?

FWIW “It is possible to convert Adobe lensprofiles using the ‘lensfun-convert-lcp’ command”
see here -

Lensfun - getting it to work with darktable

true, depends on the lens. Zeiss is superb (modern lenses like Milvus or Otus), but vignette is very present and annoying, I want to get rid of it most of the time. For wide-angle, there is still some distortion that need to be removed (sometimes complex mustache-like type, as for Zeiss 15mm). As for the procedure, thanks, I will have a look at it. Just wonder why big companies like Zeiss dont allocate ressource to profile all of their lenses in LensFun. Specialy Zeiss, who suffered from “techn…