Building darktable-master from github

Hi all,

I’m on a mission to try and build darktable from source. This has turned out to be bit more of a challenge than I expected, not least because what little documentation exists on this subject is either grossly inadequate or out of date or both. Trust me when I say I visited ALL the links you can find on Google about this. There are no instructions that work out of the box.

So I went my own way and managed to get a good part of the work done. Actually to the point of getting a success return code out of cmake and a finished binary. The problem is that the final executable is dynamically linked and contains references to dependencies I don’t have.

I guess that doesn’t make a lot of sense so let me explain how I build darktable, in case anyone else feels up to the challenge.

Step 1: Dependencies

Yes and there are SO many dependencies to have in place before you even think of starting a build. Fortunately the developers were sensible enough to use docker to setup the Travis CI build environment and we can help ourselves to the Dockerfile they use for that. All you need is to install docker on your local machine (sudo apt install docker.io I think on Ubuntu).

Go to darktable/.ci at master · darktable-org/darktable · GitHub and grab the Dockerfile from there. Now go to a command line and execute:

$ docker build - < Dockerfile
(tons of output)

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
<none>              <none>              7fb36dc89c21        2 hours ago         2.05GB

Copy the image id, you’ll need it for later. You can also name the image but for this purpose we don’t care what the image is called.

Right so now we have an image which is almost suitable for a build. I execute the following interactively but you could of course stuff all this into the Dockerfile if you wanted.

$ docker run -it <IMAGE-ID-FROM-ABOVE> bash
root@77d58544b6e4:/# apt update && apt install build-essential
(tons of output)
root@77d58544b6e4:/# mkdir github
root@77d58544b6e4:/# cd github
root@77d58544b6e4:/# git clone --depth=1 https://github.com/darktable-org/darktable.git
(tons of output)
root@77d58544b6e4:/# cd darktable
root@77d58544b6e4:/# git submodule init && git submodule update

Step 2: Compiling darktable

Finally we’re ready to build. We do it the easy way, as per the instructions on GitHub - darktable-org/darktable: darktable is an open source photography workflow application and raw developer

root@77d58544b6e4:/# ./build.sh --prefix /opt/darktable --build-type Release

Now go grab yourself a cup of coffee, this will take a while. Lots of output truncated here for brevity, but you will see something like this at the end:

[100%] Linking CXX shared module libdemosaic.so
[100%] Built target demosaic
darktable finished building.
To actually install darktable you need to type:
$ cmake --build "/github/darktable/build" --target install -- -j8

Step 3: Building the final binary

Right so we do what the build script kindly asks us to:

root@77d58544b6e4:/# cmake --build "/github/darktable/build" --target install -- -j8

(lots of output)

Now, witness with amazement the built binary of darktable:

root@77d58544b6e4:/# ls -la /opt/darktable/bin
total 25244
drwxr-xr-x 2 root root     4096 Jan 22 18:40 .
drwxr-xr-x 5 root root     4096 Jan 22 18:40 ..
-rwxr-xr-x 1 root root    25056 Jan 22 18:40 darktable
-rwxr-xr-x 1 root root   466376 Jan 22 18:40 darktable-chart
-rwxr-xr-x 1 root root    97704 Jan 22 18:40 darktable-cli
-rwxr-xr-x 1 root root    36328 Jan 22 18:40 darktable-cltest
-rwxr-xr-x 1 root root    52816 Jan 22 18:40 darktable-cmstest
-rwxr-xr-x 1 root root    49792 Jan 22 18:40 darktable-generate-cache
-rwxr-xr-x 1 root root 25100304 Jan 22 18:40 darktable-rs-identify

Okay. Now we finally get to the crux of the problem, namely that I want to copy that sucker out of the docker container and use it on my host machine. Think that will work? Let’s try it.

Open another terminal window. Note: DO NOT EXIT the container! If you’re not familiar with Docker, that container and everything in it pretty much disappears when you exit it. You’d have to start over with the build process.

(note: substitute your own container id, it’s the big number in the command prompt from inside the container, or find it with “docker container ps”)

hostmachine$ docker cp 77d58544b6e4:/opt/darktable/bin .

Great, we have it now. Let’s run it:

hostmachine$ bin/darktable
bin/darktable: error while loading shared libraries: libdarktable.so: cannot open shared object file: No such file or directory

Ah. Yeah. Bummer. The thing is obviously dynamically linked and on the host we have none of those dependencies. You can actually grab the libdarktable.so from the lib folder of the build as well, but that just leads to the next missing dependency (libIlmImf-2_3.so.24).

So. Now you know where I’m at. Is there a way to build darktable with statically linked objects? I tried using LDFLAGS="-static" on the cmake commands but that didn’t work - it was a long shot of course.

If anyone can help me along so I can run any branch or tag of darktable from source I’d appreciate it very much.

Regards,
Michael

I actually found it surprisingly straightforward to build on Fedora linux.

I just followed the instructions on the dependencies on the “comprehensive list” link, and then the “easy way” instructions on the README.

I then just had to add the destination directory to my PATH, and create symbolic links for desktop integration.

sudo ln -s /opt/darktable/share/applications/darktable.desktop /usr/share/applications/darktable.desktop
sudo ln -s /opt/darktable/share/icons/hicolor/scalable/apps/darktable.svg /usr/share/pixmaps/darktable.svg

There is packages for most linux distributions. why not use them?

to solve your problems:
you need to copy out more than just the binary. just extract the whole /opt/darktable path to your host system.

Really? That sounds great! Can you point me to a distribution or just a ppa containing the master branch from github, so I can test out new features as they are committed? That’s the whole reason I am on this adventure so if you know of a PPA for daily builds then let’s have it.

Manjaro/KDE AUR also has the latest.

Have fun!
Claes in Lund, Sweden

@bdarcus Thanks for that link, I did come across that as well. It bothered me quite a bit that these instructions were very old and clearly written for DT 2.0 which was new like, in the previous millennium or something. I figured that all those dependencies were superseded by generations of versions since that time. I don’t want to end up with a darktable which is too different from the version being built by the DT team and their CI server.

But I’ll give it another try.

@Claes Ubuntu has a 3.0 PPA as well.

Perhaps I’m not expressing myself clearly enough here. I’m not looking for a 3.0 release build, I want to try to build Darktable from source so I can test out new fixes and features as they are committed.

Some distribution X may very well have a repo for DT 3.0 but that doesn’t help me very much in my mission here.

That’s what I meant.
You’ll get the up-to-date git version. (But Manjaro will compile it for you.)

@Claes Ohh. I didn’t realise that. I’m on Ubuntu and not very knowledgeable about Manjaro.

I’m not quite sure how to proceed. Maybe spin up a VirtualBox, install Manjaro and get it to build it for me using this AUR system whatever it is.

Just seems like a roundabout way to do it. I appreciate the tip but I’ll keep hacking this on my local machine a bit longer.

Do you have a sufficiently large USB-stick?
Then you could run a live (Manjaro) version from that.

If you want to continue to use the docker container, you need to allow the container to write data to the host system using volumes: Use volumes | Docker Documentation

I wondered about that, but just tried it, and it worked. The dependency list doesn’t specify specific versions, so you end up with the latest versions.

1 Like

@paperdigits That’s not necessary. I don’t want the container to write anything to the host. I might as well not use Docker at all then.

@darix Thanks but this page does not tell me anything useful about how to build darktable from source.

Being able to build from source imho is a major requirement for foss. RT devs spent a lot of time to make it easy to build RT from source. Would be good if building dt from source also would be that easy. Tough I have to admit, that I did not try to build dt from source…

I also do not have any problems building darktable, Rawtherapee and Art from sources on Fedora 31. I do it almost daily to see what’s new there.

2 Likes

You have a build manual for Ubuntu 18.04 here : How to build the manual

you can just install master from packages there.

@rgo Thanks, this looks very interesting. Much more up to date information. It probably doesn’t help me produce a statically linked version of darktable but it’s very nice to have updated build information, so thanks for that.