C++ Lensfun Database Routine/Program

Software
1

I’m getting rawproc to a 1.0, and one of the things I want to fully support is lens correction. I have a new camera with a lens that is apparently designed to trade distortion for other qualities, and distortion correction is actually hard-coded into it’s Lightroom processing. Seems to be the way of the world, these days; software will save us all… :face_with_raised_eyebrow:

Anyway, to support lensfun integration I wrote a hacky C++ function to do what the update-lensfun-database utility does so I could include the function in some rawproc menu invocation. Surrounded it with 25 lines of main (), and tink-poof, got a program. Thinking it to be useful, or at least humorous, I’ve posted a Github repository here:

GPL2-licensed, like rawproc (if you look real soon, it may be before I post the license file, have to find a copy somewhere…). You can fork it, copy it, screen-scrape it, whatever, just use it under GPL2 terms.

It needs libcurl and libarchive. It almost needed some JSON parser, but I decided to just do a stupid-std::string-pet-trick to parse the versions.json file you need to figure out what to download next. I inserted step-by-step comments to document the sequence of operations, so one might just study that and write their own thing, fine with me.

It compiles nicely with gcc, and the MSYS2 flavor, and the Makefile is set up accordingly. There’s no install target, I left that as an exercise for the student.

Anyhoo, have fun!

5 Likes
2

I made some changes, mainly to support incorporation in rawproc. There are now two functions, one to check the database against the server, and one to do the actual update. I’ve now incorporated it in rawproc, with the first function informing the About dialog alongside the lensfun database path specified in the properties, and in an “Updata data…” menu selection, where I’ll eventually add update of other things like camera data (camconst.json from RT) and ICC profiles (@Elle’s well-behaved profiles). Thinking this would be the best way to support the Appimage and wininstallers, rather than packing dated data in them…

If nothing else, food for thought…

1 Like
3

I built the app without problem and copied the exe inside an ART build.
edit: I modified the main program to have version 2 DB and to fetch in the relative path share/lensfun where the DB is located.
It fails as it creates a /version_2 subdir.
So two questions:

  • Would it be possible to specify the relative path of the ART shipped DB which is “share/lensfun” without the creation of a /version_x subdir?

  • would it be possible to automatically check for the lensfun DB version without explicitly giving it?

Thanks

4

Ah, so the dbupdate.cpp doesn’t make full use of the library. I’ll expand it to allow specification of the path (and URL of the repository, if one had a different source than lensfun.sourceforge.net).

As the library is currently coded, if the programmer doesn’t specify a path then it works relative to the CWD of the running program, so relative paths are supported in the library.

I incorporated the library in rawproc, but I had to make a few modifications to exclude duplicate support routines, and then I made some other improvement modifications that I haven’t yet backported to the lensfun_dbupdate repo. I’ll do all of this forthwith…

5

In ART (and RT) windows builds, the Lensfun DB is located in share/lensfun. There is no version_2 subdir.

so I modified dbupdate.cpp as follows:

#include
#include “lensfun_dbupdate.h”

//print error and exit:

void err(std::string msg)
{
printf(“Error: %s\n”,msg.c_str());
exit(0);
}

int main(int argc, char ** argv)
{
int dbversion = 1;
if (argc >= 2) dbversion = atoi(argv[1]);

switch (lensfun_dbupdate(2, “share/lensfun”)) {
case LENSFUN_DBUPDATE_OK: std::cout << “database updated” << std::endl; break;
case LENSFUN_DBUPDATE_NOVERSION: std::cout << “no version available” << std::endl; break;
case LENSFUN_DBUPDATE_CURRENTVERSION: std::cout << “local database is latest” << std::endl; break;
case LENSFUN_DBUPDATE_RETRIEVFAIL: std::cout << “database retrieve failed” << std::endl; break;
}

}

as I don't want to transmit arguments to the exe., I

With above program, a ./share/lensfun/version_2 subdir is added and updated, so it fails to update ./share/lensfun.

What I hope:

  • the database is fetched and updated in ./share/lensfun and not ./share/lensfun/version_2
  • a plus should be that the version of DB be automatically identified.

edit with this hack, I think I can get what I want (still to test thouroughly).

Bug: the timestamp is updated but not the xml files

diff --git a/dbupdate.cpp b/dbupdate.cpp
index 55e9278..f3d3e6b 100644
--- a/dbupdate.cpp
+++ b/dbupdate.cpp
@@ -15,7 +15,7 @@ int main(int argc, char ** argv)
 	int dbversion = 1;
 	if (argc >= 2) dbversion = atoi(argv[1]);
 	
-	switch (lensfun_dbupdate(dbversion)) {
+	switch (lensfun_dbupdate(2, "share/lensfun")) {
 		case LENSFUN_DBUPDATE_OK: std::cout << "database updated" << std::endl; break;
 		case LENSFUN_DBUPDATE_NOVERSION: std::cout << "no version available" << std::endl; break;
 		case LENSFUN_DBUPDATE_CURRENTVERSION: std::cout << "local database is latest" << std::endl; break;
diff --git a/lensfun_dbupdate.cpp b/lensfun_dbupdate.cpp
index 17c9900..3c712ce 100644
--- a/lensfun_dbupdate.cpp
+++ b/lensfun_dbupdate.cpp
@@ -348,8 +348,8 @@ lf_db_return lensfun_dbupdate(int version, std::string dbpath)
 	const std::string repositoryurl = "http://lensfun.sourceforge.net/db/";
 
 	//build the dir to store the lensfun database:
-	std::string dbdir = string_format("version_%d",dbversion);	
-
+//	std::string dbdir = string_format("version_%d",dbversion);	
+    std::string dbdir = string_format("",dbversion);
 	//get versions.json:
 	std::string versions = getAsString(string_format("%sversions.json",repositoryurl.c_str()));
6

To do this, I’d have to compile/link to lensfun.h/-llensfun, and I’m trying to stick to the lensfun-update-database protocol. Based on all that, if you were to use dbupdate, I think it’d be prudent for the calling application to tell it what version it expects to use at that location.

Thinking about adding a noversion bool to the function signatures, and using that to make or not make the version_x directory, and unpack the tar.bz2 accordingly.

7

This looks VERY nice… Very nice indeed. Won’t mind if I do use it for something…

8

I’m changing it up a bit based on @gaaned92’s posts, I’ll announce when I’ve pushed the changes.

9

@ggbutcher

  • See post above, the timestamp is updated in ./share/lensfun, but not the xml files.

  • is the path relative to where the exe is located or to the dir from where it is called?

  • if the subdir in the path don’t exist, are they created? I suppose they have to exist previously to the call to dbupdate.

  • sorry, but keeping on requiring features:
    In windows, I would like to launch the update by double clicking on the exe. But in this case, there is no visible message, so my request:
    would it be possible the app creates a very simple window where outputing the messages?

edit : and thanks for idea, sharing and answering. very appreciated.

10

in dbupdate right now, it’s the .exe’s cwd.

in dbupdate right now, it assumes the shell path where it is run, exists. In the library a path is exposed, but dbupdate isn’t using it right now. In the library right now, only the version_x directories are created if they don’t exist.

That’s a bug in the library, working on it.

I’d like to keep dbupdate a command line program. Thinking about a GUI version, you’d probably not like my rendition (wxWidgets), so I might make that an ‘exercise for the student…’ :smiley:

Actually, my original intent was for the library to be incorporated in a raw processor; dbupdate was just intended to be a demonstrator. I think if you want that sort of interactivity, incorporating lensfun_dbupdate.cpp and .h in ART would be the best way to go…

1 Like
11

understood

ok

But… on windows, the simple way to launch an exe is to double click on it (or with a shortcut) but then you get no message.
So I will create a .bat to encapsulate the .exe. It will permit also to be sure that the base dir is the dir where the . bat is located.

and command line program is for geeks like you and linux users :wink: :crazy_face:

12

I just spent a couple of hours building logic to unpack “in-place”, only to have that code trash my uncommitted up dates when I inadvertently ran it in the source directory… :cry:

I did put in logic to read path and url from the command line, and ‘./dbupdate help’ prints a nice usage message. I’m out of time today, but I’ll think about your integration needs this evening.

If you’re going to write a containing .bat file, you might just have dbupdate run, then copy all the files in version_x/ to your intended destination and delete version_x in the batch file.

And, I encourage you and @agriggio to consider just using the two functions directly in ART. In rawproc, I use lensfun_dbcheck() in my About box construction, where I provide the status of the database, and lensfun_dbupdate() in a menu handler for Edit->Data Update. Right now, that menu selection just calls the lensfun_dbupdate(), but I’m thinking of expanding it to a dialog box that allows the user to update other things, like camconst.json from RT/github, and elle’s profiles from her repo.

Here are links to appropriate code in rawproc:

About box: rawproc/rawprocFrm.cpp at master · butcherg/rawproc · GitHub

Menu handler: rawproc/rawprocFrm.cpp at master · butcherg/rawproc · GitHub

1 Like
13

To be sure that the CWD is the directory where the exe is located and not from where it is called, I need a .bat.
For now it is ok, waiting for a clean integration by devs.

14

@gaaned92, I just pushed a commit to the repo that has the following:

  • lensfun_dbupdate.cpp: an ‘inplace’ version of the lensfun_updatedb() function
  • dbupdate: running the program with no parameters looks up the lensfun database version in the lensfun library (lensfun.h), downloads and installs that version in the cwd, no version_x subdirectory.
  • To support the above so dbupdate doesn’t trash other files in the directory (ask me how I know…), the filelist() function now takes a pattern param, called with “*.xml”

Let me know if this works for you, or you find bugs…

2 Likes
15

Thanks @ggbutcher
Not at home until sunday so not able to test

edit : at home
dbupdate.exe must be in the root dir to get access to dlls
So I have to write a dbupdate.bat to go to the share/lensfun db and call the exe:

@echo off
SET var=%~dp0
PUSHD %var%\share\lensfun
echo %cd%
"%var%\dbupdate.exe" 
set /p asd="Hit enter to continue"

When building, I will copy this dbupdate.bat and dbupdate.exe in the root dir and its ok.
To update, the user has to double click on dbupdate.bat (not complicated)

Thanks for effort

16

Hi,

Is there a dpupdate.exe available for downloads to update the lensfun database in Windows? If yes, would you mind to share the exe file and have some instruction.

Thanks in advance.

17

Right now, all I have is the source code. I’m in the middle of making rawproc 1.0, and what I’d really like to do with dbupdate is make a static link executable so one wouldn’t have to worry libraries, which would take a bit of time. Probably right after New Years…

2 Likes
18

Yes here you will find “dbupdate.zip” and “Lensfun DataBase Update .md”
https://keybase.pub/gaaned92/ART-W64NightlyBuilds/

The .bat is adapted to ART, so perhaps you will have to modify it.
I think I packed the dependencies. Let me know.
Its is shipped with my ART builds, but I doubt if somebody ever used it

19

And that would be safer.
Is it possible now to make a static link?

20

Yes, I’m going to compile it with my static mxe toolchain, x86_64…