vkdt dev diary, pt2

actually pkg-config is in the buildenv. and then i manually do the pkg-config lines that get called from the makefile, it also works.

[abuild@532908e83710 ~]$ pkg-config --libs glfw3 freetype2
-lglfw -lfreetype 

…actually looking at the fedora logs on OBS now there are a few extra warnings switched on that i could work on.

Tumbleweed might have a few more as well :smiley:

Thanks, this is great! What about filmic rgb? Does it also map to filmcurv, or are there major differences?

conceptually filmic maps to filmcurv too, but i don’t understand enough of the technical details in filmic to be able to compare it to the display transform in vkdt.

one more update with somewhat of a breaking change for some hopefully oddball settings: i changed the 2d radial basis function in the colour module to a 3d one. originally i was following the 2d idea from a google paper, but came to the conclusion that (a) the rbf with the linear part will figure out a simple relationship and keep brightness linear and (b) not considering changes in brightness between say camera rgb and rec2020 is too much of a restriction to cover all cases.

i opted for the more general version now, i hope i won’t regret it :slight_smile: in any case i wanted to push this before a release so i won’t have to break legacy graph configs or deal with updates.

2 Likes

I feel like in some back and forth during the sigmoid development that AP broke out the math in that thread so @garibaldi if you sift back that very long thread you might find the filmic explanation and then have a reference for @hanatos to comment that is if for some reason this is an itch that you need to scratch… :slight_smile:

i removed the atomic library again. now fedora says:

[  109s] + /usr/lib/rpm/check-rpaths
[  109s] *******************************************************************************
[  109s] *
[  109s] * WARNING: 'check-rpaths' detected a broken RPATH OR RUNPATH and will cause
[  109s] *          'rpmbuild' to fail. To ignore these errors, you can set the
[  109s] *          '$QA_RPATHS' environment variable which is a bitmask allowing the
[  109s] *          values below. The current value of QA_RPATHS is 0x0000.
[  109s] *
[  109s] *    0x0001 ... standard RPATHs (e.g. /usr/lib); such RPATHs are a minor
[  109s] *               issue but are introducing redundant searchpaths without
[  109s] *               providing a benefit. They can also cause errors in multilib
[  109s] *               environments.
[  109s] *    0x0002 ... invalid RPATHs; these are RPATHs which are neither absolute
[  109s] *               nor relative filenames and can therefore be a SECURITY risk
[  109s] *    0x0004 ... insecure RPATHs; these are relative RPATHs which are a
[  109s] *               SECURITY risk
[  109s] *    0x0008 ... the special '$ORIGIN' RPATHs are appearing after other
[  109s] *               RPATHs; this is just a minor issue but usually unwanted
[  109s] *    0x0010 ... the RPATH is empty; there is no reason for such RPATHs
[  109s] *               and they cause unneeded work while loading libraries
[  109s] *    0x0020 ... an RPATH references '..' of an absolute path; this will break
[  109s] *               the functionality when the path before '..' is a symlink
[  109s] *          
[  109s] *
[  109s] * Examples:
[  109s] * - to ignore standard and empty RPATHs, execute 'rpmbuild' like
[  109s] *   $ QA_RPATHS=$(( 0x0001|0x0010 )) rpmbuild my-package.src.rpm
[  109s] * - to check existing files, set $RPM_BUILD_ROOT and execute check-rpaths like
[  109s] *   $ RPM_BUILD_ROOT=<top-dir> /usr/lib/rpm/check-rpaths
[  109s] *  
[  109s] *******************************************************************************
[  109s] ERROR   0001: file '/usr/libexec/vkdt/modules/i-mlv/libi-mlv.so' contains a standard runpath '/usr/lib64' in [/usr/lib64]
[  110s] error: Bad exit status from /var/tmp/rpm-tmp.svEZDG (%install)

and none of my build system contains an rpath. so i’m assuming fedora adds these by itself and then complains about exactly that. @darix would your obs magic be powerful enough to set this QA_RPATHS variable accordingly so fedora makes itself happy again?

so we did some debugging with @asn … this seems to be a bug with clang when using -fopenmp it seems to automatically set an rpath in that case. are you actually using openmp in that file? the RUNPATH is set so that the binary finds the libomp.so.X when it is in non standard locations. sadly it seems to also set it if it is in the standard location. This does not happen on openSUSE it seems.

p.s.: while you are at it … the compiler whines about a buffer overflow in that file.

yeah i suspected as much. thanks for looking into it the two of you! the mlv reading code is pretty much directly the magic lantern/mlv-app code and uses openmp for something. probably not a very good idea if i change it much (rather change upstream and re-import).

and yes if possible i’d like to not make everything single threaded for all distros because of a bug in fedora’s toolchains…

1 Like

battle testing the colour radial basis function mapping functionality in the colour module. this time for monochrome conversion.

messed with colours a fair bit (started with colourcube.pst preset and set all reds to white and all greens to black):

rolling back the history stack for comparison with the original:

final monochrome render with bright blossoms:

for comparison the straight desaturated version:

1 Like

in darktable macports based build description cmake uses following:

-DOpenMP_C_INCLUDE_DIR=/opt/local/include/libomp 
-DOpenMP_CXX_INCLUDE_DIR=/opt/local/include/libomp 
-DCMAKE_LIBRARY_PATH=/opt/local/lib/libomp

so you need to include that in the makefile invoking cmake for rawspeed

Thank you @MStraeten. Good idea. Compiling DT works on the same machine. Yet, I have no clue where to put these lines in case of vkdt. @hanatos any idea?

Thanks a lot in advance.

Guten Rutsch!

vkdt/ext/Makefile seems to contain the rawspeed cmake statement line 15 and 18

1 Like

I will give it a try. Thanks a lot.

@MStraeten Thanks. It worked.

Now, it turned out that the implementation/specification of sendfile (see fs.h) is different between Linux and macOS (4 parameters when calling vs. 6 parameters):

on macOS X:

int sendfile(int fd, int s, off_t offset, off_t *len, struct sf_hdtr *hdtr, int flags);

on Linux

ssize_t sendfile(int out_fd, int in_fd, off_t *offset, size_t count);

I will need to figure out how to map these two calls. I will dig deeper. Perhaps somebody has an idea.

“Sendfile” is also specified in “sys/socket.h” instead of sys/sendfile.h". At least for this different includes, the following helps:

#if __APPLE__
    #include <sys/types.h>
    #include <sys/socket.h>
    #include <sys/uio.h>
#else
    #include <sys/sendfile.h>
#endif

i’m using sendfile to copy a file (hopefully with a more efficient implementation than carrying every byte through the cpu first). that’s not the original use case for it (comes from networking, hence socket.h), and i suppose any macinotoshy implementation of the surrounding fs_copy would do here. there’s probably a convenient platform specific way of doing it.

Thanks @hanatos. I will try to figure something out. Still if someone already has a solution… is there anyone who already successfully compiled vkdt on macOS?

Otherwise as soon as I succeed I will post some howto.

1 Like

It is not a bug in the fedora toolchain, it is a clang feature which doesn’t make sense for a lsb. The fedora tools just detect this nonsense …

right. makes sense… clang probably adds this rpath for no good reason for all distros. but could we just set the envar that the output mentions? it seems the case here is one of the two least objectionable ones that could be ignored without causing real breakage?

i don’t know enough about how rpmbuild is called here to set it myself… but of course you guys striving for clean builds is honorable too. just don’t know how else to go about it.