vkdt dev diary, pt2

@hanatos It seems to be related to rawspeed.

-- Looking for OpenMP
-- Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS OpenMP_CXX_LIB_NAMES) (Required is at least version "4.5")
-- Could NOT find OpenMP (missing: OpenMP_CXX_FOUND) (Required is at least version "4.5")
CMake Error at cmake/src-dependencies.cmake:42 (message):
  Did not find OpenMP! Either make it find OpenMP, or pass -DWITH_OPENMP=OFF
  to disable OpenMP support.
Call Stack (most recent call first):
  CMakeLists.txt:213 (include)

I have no idea how I could provide the necessary flags. I am using clang.

You need llvm installed

r88 | darix | 2022-12-27 17:20:37 | e88ead9abf80455c57d19bfea724140a | 0 | 
Set a proper version number and make sure that the debs also use release number handling
1 Like

JFYI: the build failures on fedora are misleadingly pointing to libatomic. the real error is:

GUI_LDFLAGS are unset. more specifically the $(shell pkg-config --libs ...) for glfw and freetype2 are failing.

okay let me try this:

approximate correspondence to darktable modules

in general vkdt uses a linear rec2020 encoding for the intermediate colour buffers, and there is pretty much no Lab code at all (except to compute delta E for display/loss). so it’s really up to the user what will be scene-referred, by default most of the modules are (but you can of course put your display transform wherever you want).

i’m skipping input/output modules because dt doesn’t work this way (dt’s input/output is more coupled to the core than to the processing pipeline).

there are also various other things like histograms/waveforms/displays/colour pickers/noise profiling/gamut checking which are implemented as modules in vkdt and thus don’t have a direct counterpart in dt as a module.

there’s an obvious difference for drawn masks (via brush strokes/pentablet in vkdt and via parametric shapes or conditional blending in dt).

dt’s snapshots probably correspond most closely to the ab module, which compares two options in a single active graph.

and of course dt has none of the 3d rendering/movie/animation/keyframing functionality. about the rest:

darktable module vkdt module note
chromatic aberrations ca different tech
demosaic demosaic different tech, works transparently for xtrans/bayer
highlight reconstruction hilite uses inpainting by default
color lookup table colour uses same rbf code
color calibration colour use cc24 presets+colour pickers
input color profile colour can ingest DCP profiles or spectral response curves (not icc)
white balance colour DCP dual wb, finetune via rgb sliders
exposure colour, exposure
monochrome colour set saturation to zero, map colours before using rbf
color balance rgb grade
crop crop
rotate and perspective crop
sharpen deconv, eq, contrast none of them is a direct match
denoise (profiled) denoise wavelet part and noise profile application are similar (noise profiles not compatible since they are applied pre-blackpoint subtraction in vkdt)
lens correction lens simpler model that also fits fisheye and anamorphic, no lensfun
contrast equalizer eq overall similar wavelet approach but with different edge protection measures
diffuse or sharpen eq for the sharpen part, only isotropic diffusion for the blurring part
sigmoid filmcurv using jandren’s idea of applying the point process CDF, using pretty much unchanged dt UCS code by aurelien
graduated density grad
local contrast llap straight adaptation of the same local laplacian pyramid code (but in rgb, not Lab)
shadows and highlights llap same functionality, disjoint tech
vignetting vignette smoother falloff, dt’s version has a discontinuity
tone equalizer zones uses same core tech (guided filter/luminance quantisation)
framing frame
retouch wavelet, inpaint
space invaders quake

(i may have to update this with links to dt and vkdt docs and update a bit in the future)

10 Likes

uhm because there’s no pkg-config on fedora? i was under the impression that there’s something odd with their way of adding default rpaths and then complaining about it later. not sure how to fix any of this from my side.

actually pkg-config is in the buildenv. and then i manually do the pkg-config lines that get called from the makefile, it also works.

[abuild@532908e83710 ~]$ pkg-config --libs glfw3 freetype2
-lglfw -lfreetype 

…actually looking at the fedora logs on OBS now there are a few extra warnings switched on that i could work on.

Tumbleweed might have a few more as well :smiley:

Thanks, this is great! What about filmic rgb? Does it also map to filmcurv, or are there major differences?

conceptually filmic maps to filmcurv too, but i don’t understand enough of the technical details in filmic to be able to compare it to the display transform in vkdt.

one more update with somewhat of a breaking change for some hopefully oddball settings: i changed the 2d radial basis function in the colour module to a 3d one. originally i was following the 2d idea from a google paper, but came to the conclusion that (a) the rbf with the linear part will figure out a simple relationship and keep brightness linear and (b) not considering changes in brightness between say camera rgb and rec2020 is too much of a restriction to cover all cases.

i opted for the more general version now, i hope i won’t regret it :slight_smile: in any case i wanted to push this before a release so i won’t have to break legacy graph configs or deal with updates.

2 Likes

I feel like in some back and forth during the sigmoid development that AP broke out the math in that thread so @garibaldi if you sift back that very long thread you might find the filmic explanation and then have a reference for @hanatos to comment that is if for some reason this is an itch that you need to scratch… :slight_smile:

i removed the atomic library again. now fedora says:

[  109s] + /usr/lib/rpm/check-rpaths
[  109s] *******************************************************************************
[  109s] *
[  109s] * WARNING: 'check-rpaths' detected a broken RPATH OR RUNPATH and will cause
[  109s] *          'rpmbuild' to fail. To ignore these errors, you can set the
[  109s] *          '$QA_RPATHS' environment variable which is a bitmask allowing the
[  109s] *          values below. The current value of QA_RPATHS is 0x0000.
[  109s] *
[  109s] *    0x0001 ... standard RPATHs (e.g. /usr/lib); such RPATHs are a minor
[  109s] *               issue but are introducing redundant searchpaths without
[  109s] *               providing a benefit. They can also cause errors in multilib
[  109s] *               environments.
[  109s] *    0x0002 ... invalid RPATHs; these are RPATHs which are neither absolute
[  109s] *               nor relative filenames and can therefore be a SECURITY risk
[  109s] *    0x0004 ... insecure RPATHs; these are relative RPATHs which are a
[  109s] *               SECURITY risk
[  109s] *    0x0008 ... the special '$ORIGIN' RPATHs are appearing after other
[  109s] *               RPATHs; this is just a minor issue but usually unwanted
[  109s] *    0x0010 ... the RPATH is empty; there is no reason for such RPATHs
[  109s] *               and they cause unneeded work while loading libraries
[  109s] *    0x0020 ... an RPATH references '..' of an absolute path; this will break
[  109s] *               the functionality when the path before '..' is a symlink
[  109s] *          
[  109s] *
[  109s] * Examples:
[  109s] * - to ignore standard and empty RPATHs, execute 'rpmbuild' like
[  109s] *   $ QA_RPATHS=$(( 0x0001|0x0010 )) rpmbuild my-package.src.rpm
[  109s] * - to check existing files, set $RPM_BUILD_ROOT and execute check-rpaths like
[  109s] *   $ RPM_BUILD_ROOT=<top-dir> /usr/lib/rpm/check-rpaths
[  109s] *  
[  109s] *******************************************************************************
[  109s] ERROR   0001: file '/usr/libexec/vkdt/modules/i-mlv/libi-mlv.so' contains a standard runpath '/usr/lib64' in [/usr/lib64]
[  110s] error: Bad exit status from /var/tmp/rpm-tmp.svEZDG (%install)

and none of my build system contains an rpath. so i’m assuming fedora adds these by itself and then complains about exactly that. @darix would your obs magic be powerful enough to set this QA_RPATHS variable accordingly so fedora makes itself happy again?

so we did some debugging with @asn … this seems to be a bug with clang when using -fopenmp it seems to automatically set an rpath in that case. are you actually using openmp in that file? the RUNPATH is set so that the binary finds the libomp.so.X when it is in non standard locations. sadly it seems to also set it if it is in the standard location. This does not happen on openSUSE it seems.

p.s.: while you are at it … the compiler whines about a buffer overflow in that file.

yeah i suspected as much. thanks for looking into it the two of you! the mlv reading code is pretty much directly the magic lantern/mlv-app code and uses openmp for something. probably not a very good idea if i change it much (rather change upstream and re-import).

and yes if possible i’d like to not make everything single threaded for all distros because of a bug in fedora’s toolchains…

1 Like

battle testing the colour radial basis function mapping functionality in the colour module. this time for monochrome conversion.

messed with colours a fair bit (started with colourcube.pst preset and set all reds to white and all greens to black):

rolling back the history stack for comparison with the original:

final monochrome render with bright blossoms:

for comparison the straight desaturated version:

1 Like

in darktable macports based build description cmake uses following:

-DOpenMP_C_INCLUDE_DIR=/opt/local/include/libomp 
-DOpenMP_CXX_INCLUDE_DIR=/opt/local/include/libomp 
-DCMAKE_LIBRARY_PATH=/opt/local/lib/libomp

so you need to include that in the makefile invoking cmake for rawspeed

Thank you @MStraeten. Good idea. Compiling DT works on the same machine. Yet, I have no clue where to put these lines in case of vkdt. @hanatos any idea?

Thanks a lot in advance.

Guten Rutsch!

vkdt/ext/Makefile seems to contain the rawspeed cmake statement line 15 and 18

1 Like