Compiling darktable .dmg: A bump in the tracks

Morgan_Hardwood · January 11, 2018, 9:30am

@bram every release gets tagged, to see all tags:

git tag

Then to checkout a specific tag:

git checkout release-2.5.0

When you do that, you will be in ‘detached HEAD’ state. If you want to get the latest development code, you need to switch back to master before you pull:

git checkout master
git pull

bram · January 11, 2018, 9:35am

Thank you.

The BUILD.TXT mentions ‘git clone git://github.com/darktable-org/darktable.git’.

Can I just replace that with ‘git checkout release-2.4.0’?

Sorry for my questions - I have been a computer programmer in the past but I never worked with Git

Morgan_Hardwood · January 11, 2018, 9:36am

No, you first need to get this source code repository, and that is done by the command you quoted.
Good stuff:
https://git-scm.com/book/en/v2

bram · January 11, 2018, 9:38am

Thx. I think I’m beginning to understand what happens. I will rtfm

LebedevRI · January 11, 2018, 12:40pm

Note that there is no such release.
All releases have even numbers.
These odd tags, as the tag description says, are not releases.

bram · January 11, 2018, 5:26pm

@LebedevRI Thank you for clarifying. This was just a try. My next goal is to create a .DMG for 2.4.0

edit: I successfully created a .DMG with release 2.4.0.

One problem left - it says 2.4.0+dirty in the top left corner of the window.

I need to remove the ‘+dirty’.

But how?

Otherwise - I am happy. Now I can finally process Fuji compressed RAW

Thank you to the devs.

Hevii_Guy · January 11, 2018, 10:07pm

Bram, did you happen to do anything differently than the instructions for gtk-mac-bundler state? As noted, I was able to compile dt 2.4.0 for my Mac box but I wasn’t able to produce a functioning Disk Image.

bram · January 11, 2018, 10:21pm

No I did not. BTW I have made a pull request on GitHub with the changes I made to the BUILD.TXT file. Even though it isn’t accepted yet you can still read my changes in the pull request.

Morgan_Hardwood · January 11, 2018, 10:21pm

Maybe this can help:

github.com

Beep6581/RawTherapee/blob/dev/tools/osx/macosx_bundle.sh#L206

    
      
          
          
# Copy the Lensfun database into the app bundle
          mkdir -p "${RESOURCES}/share/lensfun"
          ditto ${LOCAL_PREFIX}/local/share/lensfun/version_2/* "${RESOURCES}/share/lensfun"
          
          
# Copy liblensfun to Frameworks
          ditto ${LOCAL_PREFIX}/local/lib/liblensfun.2.dylib "${CONTENTS}/Frameworks/liblensfun.2.dylib"
          
          
# Copy libomp to Frameworks
          ditto ${LOCAL_PREFIX}/local/lib/libomp.dylib "${CONTENTS}/Frameworks"
          
          
# Prepare GTK+3 installation
          msg "Copying configuration files from ${GTK_PREFIX}:"
          install -d "${ETC}/gtk-3.0"
          msg "Copying library modules from ${GTK_PREFIX}:"
          ditto --arch "${arch}" {"${GTK_PREFIX}/lib","${LIB}"}/gdk-pixbuf-2.0
          ditto --arch "${arch}" {"${GTK_PREFIX}/lib","${LIB}"}/gtk-3.0
          msg "Removing static libraries and cache files:"
          find -E "${LIB}" -type f -regex '.*\.(a|la|cache)$' | while read -r; do rm "${REPLY}"; done
          
          
# Make Frameworks folder flat

houz · January 17, 2018, 1:17pm

Parafin is back and said that the OSX package will be build next weekend.

Hevii_Guy · January 17, 2018, 1:30pm

Welcome back, Parafin

homer3018 · January 27, 2019, 3:11pm

Hi there,
I’m new to Mac and as I’d like to contribute to DT and be able to pull the latest dev version, compile it and use it along side 2.6.0 (is that ok?) I’m trying to follow the BUILD.TXT instructions.

I’m currently stuck with this line :cp -R "$(port dir python36)" ~/ports/lang
which is part of all the patches for python and other things just because we’ve set the deployment target to 10.7.
I’m totally unsure as to know if this is absolutely necessary… This is what I get:

$ cp -R "$(port dir python36)" ~/ports/lang
-bash: port: command not found
cp: : No such file or directory

Edit : I haven’t restarted my terminal after installing Macports hence this error… Sorry about that, just figured that out. I am still going through the build.txt and will let you know where I’m at

Thanks everyone
Cheers

homer3018 · January 28, 2019, 10:14pm

Hi again,

I’ve got further following the build instructions and was able to install a lot of packages and dependencies with

$ sudo port install git exiv2 libgphoto2 gtk-osx-application-gtk3 lensfun librsvg libsoup openexr json-glib flickcurl GraphicsMagick openjpeg lua webp libsecret pugixml osm-gps-map adwaita-icon-theme tango-icon-theme intltool iso-codes libomp

However it’s stopped on GNUTLS :

—> Building gnutls

Error: gnutls cannot be built while autoconf-archive is active.

Error: Please forcibly deactivate autoconf-archive, e.g. by running:

Error:

Error: sudo port -f deactivate autoconf-archive

Error:

Error: Then try again. You can reactivate autoconf-archive again later.

Error: Failed to build gnutls: autoconf-archive is active

Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gnutls/gnutls/main.log for details.

Error: Follow MacPorts Guide to report a bug.

Error: Processing of port libsoup failed

I’ve tried what’s suggested, and also deactivate gnome-common but still failing in the middle of it :

$ sudo port install gnutls

Password:

—> Computing dependencies for gnutls

—> Building gnutls

Error: Failed to build gnutls: command execution failed

Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gnutls/gnutls/main.log for details.

Error: Follow MacPorts Guide to report a bug.

Error: Processing of port gnutls failed

Here’s the end of the log if it helps. I can’t figure out what’s wrong with it…

:info:build ld: warning: cannot export hidden symbol __gnutls_x86_cpuid_s from accelerated/.libs/libaccelerated.a(x86-common.o)
:info:build ld: weak import of symbol ‘_connectx’ not supported because of option: -no_weak_imports for architecture x86_64
:info:build clang: error: linker command failed with exit code 1 (use -v to see invocation)
:info:build make[4]: *** [libgnutls.la] Error 1
:info:build make[4]: Leaving directory /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gnutls/gnutls/work/gnutls-3.6.6/lib' :info:build make[3]: *** [all-recursive] Error 1 :info:build make[3]: Leaving directory /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gnutls/gnutls/work/gnutls-3.6.6/lib’
:info:build make[2]: *** [all] Error 2
:info:build make[2]: Leaving directory /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gnutls/gnutls/work/gnutls-3.6.6/lib' :info:build make[1]: *** [all-recursive] Error 1 :info:build make[1]: Leaving directory /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gnutls/gnutls/work/gnutls-3.6.6’
:info:build make: *** [all] Error 2
:info:build make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gnutls/gnutls/work/gnutls-3.6.6’
:info:build Command failed: cd “/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gnutls/gnutls/work/gnutls-3.6.6” && /usr/bin/make -j8 -w all
:info:build Exit code: 2
:error:build Failed to build gnutls: command execution failed
:debug:build Error code: CHILDSTATUS 79587 2
:debug:build Backtrace: command execution failed
:debug:build while executing
:debug:build “system {}$notty {}$nice $fullcmdstring”
:debug:build invoked from within
:debug:build “command_exec build”
:debug:build (procedure “portbuild::build_main” line 8)
:debug:build invoked from within
:debug:build “$procedure $targetname”
:error:build See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_devel_gnutls/gnutls/main.log for details.

Thanks ahead of time for your support, really appreciated !

MStraeten · January 29, 2019, 6:31am

Due to gnutls updated behaviour parafin changed the build description.
You need to perform (copies from BUILD.txt)
mkdir -p ~/ports/devel/gnutls/files
cp -R “$(port dir gnutls)” ~/ports/devel
curl -Lo ~/ports/devel/gnutls/files/patch.diff https://raw.github.com/darktable-org/darktable/master/packaging/macosx/gnutls-disable-connectx.diff

then append this line:
patchfiles-append patch.diff
to ~/ports/devel/gnutls/Portfile files you just copied (except for pugixml) and run:
$ portindex ~/ports

and then wait for a rawspeed patch for openmp support, that is already implemented in the rawspeed project but not yet transfered to darktables submodule directory - or copy the changed files from rawspeed project yourself.

homer3018 · January 29, 2019, 9:41pm

Thanks !!
I applied the patch and then everything ran smoothly from there, and was able to go through

$ sudo port install git exiv2 libgphoto2 gtk-osx-application-gtk3 lensfun librsvg libsoup openexr json-glib flickcurl GraphicsMagick openjpeg lua webp libsecret pugixml osm-gps-map adwaita-icon-theme tango-icon-theme intltool iso-codes libomp

entirely. Indeed I see more recent commits on rawspeed dealing with openmp (like 369c170), and on DT it seems that the most recent commit in src/external/rawspeed is 5 days old (1ca1601).
The latter seems to be the one I need to consider.

Thanks again for helping

homer3018 · February 3, 2019, 4:10pm

Hello all,
Thanks to all the advices I’ve managed to build my very first DT .dmg file. I installed it just fine and now I can run DT 2.6.0 or current build 2b76ca3.

For those interested in the details, I’d be happy to discuss further what I did and what happened.

The only thing that remain unsolved is the very last line of the BUILD.TXT file where it says, before a new build, to uninstall DT from /usr/local assuming it is the only program installed there. This is not the case for me, so I’m not quite sure how the proceed there.

Would only remove /usr/local/share be enough?

Thanks again for helping me out

Hevii_Guy · February 3, 2019, 4:57pm

Indeed, Seb; a detailed description of the steps taken, problems encountered and corrections /adjustments made to the procedure would be highly appreciated.would be much appreciated

homer3018 · February 3, 2019, 6:42pm

Ok, here we go. I’ll try to make it as accurate and concise as I can.

First, since I git clone --recurse-submodules https://github.com/darktable-org/darktable.git last week, I needed to make sure I had everything updated. so I

$ cd ~/src/darktable
$ git pull

and then resume the built.txtinstructions :

 $ cd darktable
 $ mkdir build
 $ cd build
 $ cmake .. -DCMAKE_OSX_DEPLOYMENT_TARGET=10.7 -DCMAKE_C_FLAGS=-I/opt/local/include/libomp -DCMAKE_CXX_FLAGS="-stdlib=libc++ -I/opt/local/include/libomp" -DCMAKE_LIBRARY_PATH=/opt/local/lib/libomp -DBINARY_PACKAGE_BUILD=ON -DRAWSPEED_ENABLE_LTO=ON
 $ make

the cmake command went through but complained that I haven’t had JDK installed and suggested a link. I went ahead and downloaded the dmg file here.
the make failed also at 7% while building src⁩ ▸ ⁨darktable⁩ ▸ ⁨src⁩ ▸ ⁨external⁩ ▸ ⁨rawspeed⁩ ▸ ⁨src⁩ ▸ ⁨librawspeed⁩ ▸ ⁨common⁩ ▸ ⁨Rawimage.cpp⁩. I opened it and realised that I was not using the latest commit which included the GnuTLS update. In fact my first git pull did update DT repo but not its submodules. So this is when I :

 $ git submodule update --remote —recursive

although I think that

$ cd ~/src⁩/darktable⁩/src⁩/external⁩/rawspeed
$ git pull

would have done the trick as well. (Can someone confirm that ?) I guess I’m not familiar enough with working with git and submodules.
Checking the hash number and Rawimage.cpp confirmed that I’ve now updated everything correctly. I then did :

$ cd ~/src⁩/darktable
$ rm -r build
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_OSX_DEPLOYMENT_TARGET=10.7 -DCMAKE_C_FLAGS=-I/opt/local/include/libomp -DCMAKE_CXX_FLAGS="-stdlib=libc++ -I/opt/local/include/libomp" -DCMAKE_LIBRARY_PATH=/opt/local/lib/libomp -DBINARY_PACKAGE_BUILD=ON -DRAWSPEED_ENABLE_LTO=ON

The output of cmake looked better with JDK installed, and this time the make command went all the way to 100%.

I then did :

2). Download, patch and install gtk-mac-bundler (assuming darktable was cloned into ~/src directory):
     $ cd ~/src
     $ curl -O https://ftp.gnome.org/pub/gnome/sources/gtk-mac-bundler/0.7/gtk-mac-bundler-0.7.4.tar.xz
     $ tar -xf gtk-mac-bundler-0.7.4.tar.xz
     $ cd gtk-mac-bundler-0.7.4
     $ patch -p1 < ../darktable/packaging/macosx/gtk-mac-bundler-0.7.4.patch
     $ make install

which ran ok, followed by

3). Now preparation is done, run image creating script, it should create darktable-<VERSION>.dmg in current (packaging/macosx) directory:
     $ cd ~/src/darktable/packaging/macosx
     $ ./make-app-bundle

which failed at $ ./make-app-bundle because I’ve had skip (intentionally) the $ sudo make installon purpose. I realised that I needed it installed so the DMG can be generated. At that point I ran :

$ cd ~/src/darktable/build
$ sudo make install

This went well. So I tried again to :

     $ cd ~/src/darktable/packaging/macosx
     $ ./make-app-bundle

At that point in time it asks me if Terminal could take over Finder. I was not sure and clicked “no” or something meaning no anyway. The DMG was created but size was ridiculously small and output showed some error like “Not authorized to send Apple events to System Events.”. I knew this has to do with the authorisation I had just denied so I googled a bit and then went to setings/Security and privacy/Privacy and selected Automation on the right hand side. I could there see Terminal and just below indented Finder unchecked. Checked it, restarted Terminal and that was it for the authorisation.

Restarting Terminal and

     $ cd ~/src/darktable/packaging/macosx
     $ ./make-app-bundle

did not do the trick. The DMG file has to be deleted first. Once I’ve done that and tried once more

$ ./make-app-bundle

It works just fine and the DMG file was finally created with an approx size of 70mb. Double clicked on it (renamed the former Darktable in app as Darktable_26, and then dragged over the new Darktable.

Firing up one or the other just start the correct version of the software, and all the settings and styles that I had set up in 2.6 were in the built version also, which is great.

So to summarise I’d say that the main takeaways of this are:

delete the build directory if you’re unsure, and restart the process from there
do not forget do update the dependencies
Install JDK as it’s surely needed
Allow Terminal to take over Finder when prompted (first time only)

Yet to try :

update Darktable to the latest changes and redo the whole process
finding out how to uninstall ONLY Darktable in usr/local as it is suggested to do so before attempting another building process.

Hope this clarifies things up and helps.

Cheers

Hevii_Guy · February 3, 2019, 8:43pm

Thanks, Seb

MStraeten · February 4, 2019, 10:45am

./make-app-bundle only requires to delete the *.dmg if no new version flag was received via git pull … because the version is part of the dmg filename