Dockerfile builds HDRMerge from Linux / running from macOS

solved
docker
qt

#1

I’ve been experimenting with a Dockerfile auto-built container of HDRMerge based on Linux (Alpine didn’t work because of a libraw/memset() related error)


At first, trying to get it to build and run on mac was successful except I didn’t see any text. Got it to work though using alpine:edge with qt at 5.10. To try it:
docker pull kd6kxr/hdrmerge

My launching commands (This is done in two terminals) are
open -a xquartz && xhost + 192.168.0.100
socat TCP-LISTEN:6000,reuseaddr,fork UNIX-CLIENT:\"$DISPLAY\"
and in the second terminal:
docker run -e DISPLAY=192.168.0.100:0 -e QT_XKB_CONFIG_ROOT=/usr/share/X11/xkb -v /tmp/X11-unix=/tmp/X11-unix kd6kxr/hdrmerge

Of course you would have to replace the DISPLAY=ip address above with your own.

Im getting a whole bunch of these warnings on the terminal:
QFont::setPointSizeF: Point size <= 0 (-0.750000), must be greater than 0


edit: On Debian, the text was missing unless I added xorg-server to the dependencies list.
edit: On mac this had no effect.
edit: To get text to work on mac I had to build from alpine:edge to get qt 5.10

#2

Now to figure out how to get images to the program to try it…


(Mica) #3

Use docker volumes to expose persistent storage to your container: https://docs.docker.com/storage/volumes/


#4

Thanks Mica,
I used -v ~:/hi:private to mount my home directory to /hi in the container.
Having tested a merge, I just get a Segmentation fault

Now building a debug version in my local testing dockerfile and running with gdb using docker run -it --rm -e DISPLAY=192.168.0.100:0 -e QT_XKB_CONFIG_ROOT=/usr/share/X11/xkb -e FONTCONFIG_PATH="~/.local/share/fonts/" -v /tmp/X11-unix=/tmp/X11-unix -v ~:/hi:private --cap-add=SYS_PTRACE --security-opt seccomp=unconfined test1 bash and running gdb ~/programs/hdrmerge/hdrmerge I can get a short backtrace of the problem:

Thread 9 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.

(gdb) bt
#0  0x00007ffff7dbddfc in memset () from /lib/ld-musl-x86_64.so.1
#1  0x00007ffff7aff68b in LibRaw::LibRaw(unsigned int) () from /usr/lib/libraw_r.so.16
#2  0x000055555558c4af in hdrmerge::ImageIO::loadRawImage (filename=..., rawParameters=..., shot_select=shot_select@entry=0)
    at /root/programs/code-hdrmerge/src/ImageIO.cpp:37
#3  0x000055555558cf8b in hdrmerge::ImageIO::load (this=0x7fffffffe9e0, options=..., progress=...) at /root/programs/code-hdrmerge/src/ImageIO.cpp:129
#4  0x00005555555966b2 in std::function<int ()>::operator()() const (this=0x555556610cc0) at /usr/include/c++/6.4.0/functional:2127
#5  QtConcurrent::StoredFunctorCall0<int, std::function<int ()> >::runFunctor() (this=0x555556610ca0) at /usr/include/QtConcurrent/qtconcurrentstoredfunctioncall.h:60
#6  QtConcurrent::RunFunctionTask<int>::run (this=0x555556610ca0) at /usr/include/QtConcurrent/qtconcurrentrunbase.h:108
#7  0x00007ffff5ede963 in ?? () from /usr/lib/libQt5Core.so.5
#8  0x00007ffff5ee29b2 in ?? () from /usr/lib/libQt5Core.so.5
#9  0x00007ffff7dbf6c2 in ?? () from /lib/ld-musl-x86_64.so.1
#10 0x0000000000000000 in ?? ()

(Mica) #5

You’re above my pay grade :wink:

My only question would be: can musl handle all the X11 stuff? I have no idea, but musl is generally for embedded systems. Perhaps trying a debian or ubuntu base with glibc would work better?


#6

I have faith in the alpine image as it’s apk package manager has plenty of x11 stuff in there. What I’m questioning is musl might have a small stack size (80kB) which is being overloaded. I think this can be changed in the source code, but that’s above my pay grade.


(Mica) #7

Looking at the dockerfile, in theory, it isn’t hard to switch from alpine to something else. Switch the “FROM alpine:edge” statement and change the apk add to apt install.

I’m sure you can change the stack size, but that’d mean recompiling everything or using some patched version of alpine.


#8

I might try that change, was just hoping to save some Megabytes.
Looking at the backtrack my thinking is that a code in libraw is initializing some memory, possibly passing the sizeof() of an incorrect variable type for musl-libc?


#9

More memset errors when trying libraw 0.19 beta…

Thread 9 "Thread (pooled)" received signal SIGSEGV, Segmentation fault.
[Switching to LWP 11605]
0x00007ffff7dbddfc in memset () from /lib/ld-musl-x86_64.so.1
(gdb) bt
#0  0x00007ffff7dbddfc in memset () from /lib/ld-musl-x86_64.so.1
#1  0x00007ffff7afa38e in memset (__n=345576, __c=0, __d=0x7ffff7f4d9e8) at /usr/include/fortify/string.h:70
#2  LibRaw::LibRaw (this=0x7ffff7f4d9e0, flags=0) at src/libraw_cxx.cpp:382
#3  0x000055555558c4bf in hdrmerge::ImageIO::loadRawImage (filename=..., rawParameters=..., shot_select=shot_select@entry=0)
    at /root/programs/code-hdrmerge/src/ImageIO.cpp:37
#4  0x000055555558cf9b in hdrmerge::ImageIO::load (this=0x7fffffffe980, options=..., progress=...) at /root/programs/code-hdrmerge/src/ImageIO.cpp:129
#5  0x00005555555966c2 in std::function<int ()>::operator()() const (this=0x555556693100) at /usr/include/c++/6.4.0/functional:2127
#6  QtConcurrent::StoredFunctorCall0<int, std::function<int ()> >::runFunctor() (this=0x5555566930e0) at /usr/include/QtConcurrent/qtconcurrentstoredfunctioncall.h:60
#7  QtConcurrent::RunFunctionTask<int>::run (this=0x5555566930e0) at /usr/include/QtConcurrent/qtconcurrentrunbase.h:108
#8  0x00007ffff5eaf963 in ?? () from /usr/lib/libQt5Core.so.5
#9  0x00007ffff5eb39b2 in ?? () from /usr/lib/libQt5Core.so.5
#10 0x00007ffff7dbf6c2 in ?? () from /lib/ld-musl-x86_64.so.1
#11 0x0000000000000000 in ?? ()

#10

Success with a conversion to ubuntu:latest!


One benefit of going up to `ubuntu:latest` was we don't need to patch `CMakeLists.txt` for alglib.

(Mica) #11

Awesome! From my limited understanding, musl isn’t at feature parity with glibc, and is intentionally pared down for embedded systems.


#13

The present kd6kxr/hdrmerge Ubuntu-based docker image is 413MB. Alpine was plenty smaller.
As far as alpine/musl being up to the task, I think that darktable supports alpine, and they no longer use libraw from what I gather. The darktable/darktable Dockerfile uses Debian:testing.
I’ll see what I can save with --no-install-recommends.


(Mica) #14

Indeed darktable uses rawspeed instead.


#15

The latest libraw 0.19 beta uses Rawspeed as well, dropping the GPL2 and 3 demosaicing packs also.