vkdt dev diary, pt2

I think that CR3 PR has been abandon, as Roman said he’d need to rewrite it, but he hasn’t started rewriting it yet, which is why there is an in-tree version of libraw in darktable now.

i will not comment on the libraw thing.

the canon_cr3 branch i used above is 8 days old at the time of this writing.

that sounds great! not that i have a cr3 camera or the money to get one :slight_smile:
but decoding this thing took 500ms (!!!) here. that’s like half a second! unacceptable!11

i remember some formats were easier to parallelise, for instance the fuji raws decode so much quicker. hoping the tile based wavelets in cr3 offer enough possibility for parallel processing. i’m sure if anyone can do then it’ll be you.

1 Like

One of the problems is, that the current version (though already using omp) only uses up to 4 cores. I’ll see what I can do.

this is now pushed. also i updated rawspeed to new upstream. it contains colour matrices :slight_smile: so prospectively i can delete the adobe_coeff.h header. currently it’s still in use for some other things though (mlv, external tools) that need fixing.

note that for cr3 support you’d still need to compile in cytrinox’ branch, as in the listing above.

as of today, vkdt has initial support for timelapse videos. there is some minimal documentation, too.

in short, the i-raw module now supports filenames of the form DSCF%04d.RAF (for instance) and will open a sequence of numbers as animation. this can be played back in the gui or exported to h265 video (via gui export or cli).

currently this requires some manual .cfg file editing (setting number of frames, initial frame number, frames per second, setting filename with %04d). if you’re into timelapses and give this a spin, let me know how it goes.

2 Likes

Wait, so you go to spectral and create on-the-fly CMF to produce as many virtual “primaries” as you want ?

there is some code for this, yes. i should clean this up some. happy to discuss in more detail or take your input on how to best streamline it for certain use cases.

here’s a quick timelapse test run with 10 frames, using this cfg:

frames:10
fps:10
module:i-raw:main
module:denoise:01
[...]
param:filmcurv:01:black:-4.80169
param:i-raw:main:filename:DSCF%04d.RAF
param:i-raw:main:startid:4683

only the first two and last two lines are important for timelapses. i don’t really have images for this, so i’m only running it on a short burst of 10 frames (that’s the frames:10 line above). the result can be viewed like this:

adding keyframes like this

keyframe:0:colour:01:exposure:0:1:0
keyframe:9:colour:01:exposure:0:1:4

will make the sequence go from exposure 0 (frame 0) to exposure 4 (frame 9). you can create such keyframes from the gui by stopping the animation (space or the stop button) at the desired frame, setting the values you’d like, hovering over the control, and pressing ‘k’.

running the resulting graph through the cli (the same can be achieved from gui export):

vkdt-cli -g ~/Pictures/20200702_teapot/timelapse.cfg --format o-ffmpeg --filename timelapse --output main

and reencoding the h264 to webm for this post:

ffmpeg -i timelapse_0000.h264 output.webm

i arrive at the following video:

1 Like

Wow, that looks like that’s already the best time laps tool for Linux. :slight_smile:

Two questions:
Do you have any plans to add “auto align” of the images ?
And does crop and zoom work? (To make the Ken Burns effect)

I think both is impotent for nice time laps. Both can be don in post, but it would be nice to have in one tool.

the current align module will align the full image, not only do a camera shake correction (i.e. the teapot would not be moving any more). so that’d require a new module.

ah good question. i don’t think the k for keyframe shortcut will work on the ui widget. i just tried to manually put keyframes of the crop module in the .cfg, which works “fine”. actually i think this should probably be done in a different transformation module that will never change the resolution of the output. animating output size sounds like asking for trouble (the current implementation kinda works nontheless but leaves black borders if you animate to crop smaller than frame 0 did). could be a switch in the c/r module i suppose.

Good example and cute stop motion. :+1:

1 Like

actually the lens module works kinda okay on this one. let me demonstrate with a still image:

these are the interesting lines in the .cfg:

frames:200
fps:24
keyframe:0:lens:01:center:0:2:-0.25157:0
keyframe:199:lens:01:center:0:2:0.24528:0.21384
keyframe:0:lens:01:scale:0:2:1.9:1.9
keyframe:199:lens:01:scale:0:2:2.1:2.1
keyframe:0:lens:01:squish0:0:1:0
keyframe:199:lens:01:squish0:0:1:0.9434
1 Like

quick diary entry. today: initial wacom pen/tablet support for drawn masks.

i added support for raster masks drawn with a stylus and pen pressure tied to opacity (so far). this was a bit of a journey because glfw does not yet support these pentablet events. luckily there is already a working pull request for it, so i just had to build this branch and publish it somewhere and then link vkdt against this and implement a few callbacks.

because of the non-std glfw (i hope the feature will make it to 3.4!), the vkdt code sits in a branch and requires some manual tweaking to link against the correct glfw version.

in particular, i have the following in my bin/config.mk:

# use our custom glfw from git
VKDT_GLFW_CFLAGS=-I/home/XX/vc/ext/glfw/include/
VKDT_GLFW_LDFLAGS=/home/XX/vc/ext/glfw/build/src/libglfw3.a

which points to a checkout/build of said glfw version.

i suppose i’ll merge it to master with a compile time switch/detection whether glfw supports the callbacks soon, and hope stock glfw will ship pen support at some point.

4 Likes

a bit off topic, but that’s today’s news:

pushed an optional module i-quake to render the animation from the other thread. to test, pull and

  • edit config.mk to contain the two lines about VKDT_USE_QUAKE as per instructions in config.mk.defaults
  • build vkdt, you should then have modules/i-quake/*{spv,so}
  • get and extract a blue noise texture
  • install quake as usual in /usr/share/games/quake
  • run ./vkdt examples/quake.cfg
  • press space bar to start animation, maybe grab mouse input in the i-quake module

similar info is found in the documentation for the module.

this is mostly for fun and may be useful as a testbed for image alignment, denoising, video output, and audio code.

3 Likes

Hey,
I just compiled vkdt which worked after installing a few missing dependencies. When attempting to run it for the first time I got

an error occurred while trying to execute gdb.please check if gdb is installed on your system.
backtrace written to /tmp/vkdt-bt-14646.txt

then after installing GDB I got

[gui] glfwGetVersionString() : 3.3.6 X11 GLX EGL OSMesa clock_gettime evdev shared
[gui] monitor [0] DP-2 at 3840 0
[gui] monitor [1] DP-3 at 0 0
[gui] vk extension required by GLFW:
[gui]   VK_KHR_surface
[gui]   VK_KHR_xcb_surface
[gui] no display profile file display.DP-2, using sRGB!
[gui] no display profile file display.DP-3, using sRGB!
backtrace written to /tmp/vkdt-bt-14770.txt

with vkdt-bt-14770.txt containing only this one line:

this is vkdt 0.0.1-134-g62b0c2f reporting a segfault:

My GPU might be on the very low end (Quadro K2000 with 2GB GDDR5) so this might be the issue here? I’ll get a new one eventually :wink:

heya,

could you run vkdt -d all to see if it maybe outputs another clue for me? like this and with the backtrace dumping not working it’s hard to guess what’s going on. if you didn’t even have gdb installed before i’m not going to ask you to try and run it manually :slight_smile:

ps: wow, how these expensive high end quadros become mediocre over the years is incredible. on the other hand it’s been almost ten years…

1 Like

Hey :slight_smile:

/opt/vkdt/vkdt/bin~ ./vkdt -d all                                                                                           127 ✘ 
[gui] glfwGetVersionString() : 3.3.6 X11 GLX EGL OSMesa clock_gettime evdev shared
[gui] monitor [0] DP-2 at 3840 0
[gui] monitor [1] DP-3 at 0 0
[gui] vk extension required by GLFW:
[gui]   VK_KHR_surface
[gui]   VK_KHR_xcb_surface
[qvk] dev 0: vendorid 0x10de
[qvk] dev 0: Quadro K2000
[qvk] max number of allocations -1
[qvk] max image allocation size 16384 x 16384
[qvk] max uniform buffer range 65536
[qvk] picked device 0 without ray tracing support
[qvk] num queue families: 2
[qvk] available surface formats:
[qvk] B8G8R8A8_UNORM
[qvk] B8G8R8A8_SRGB
[qvk] colour space: 0
[gui] no display profile file display.DP-2, using sRGB!
[gui] no display profile file display.DP-3, using sRGB!
[qvk] available surface formats:
[qvk] B8G8R8A8_UNORM
[qvk] B8G8R8A8_SRGB
[qvk] colour space: 0
[db] allocating 1024.0 MB for thumbnails
[qvk] error VK_ERROR_OUT_OF_DEVICE_MEMORY executing vkAllocateMemory(qvk.device, &mem_alloc_info, 0, &tn->vkmem)!
backtrace written to /tmp/vkdt-bt-2642.txt

The backtrace txt is as empty as before.
Sounds like my 2GB VRAM is not enough, right?
Actually I only bought the Quadro because it was the cheapest graphics card I could find that can drive my two 4k monitors when I wanted to sell my Vega 56 for three times as much as I bought it for. I didn’t expect the GPU shortage to last that long back then :smiley:

documentation update:

the readme.md file is now not only used to generate the website:

2022-04-25-132203_567x405_scrot

but also the bullet list for the parameters is extracted as tooltips:

planning to do the same for connectors and maybe for the global module description so you can see it in the ui when configuring the pipe/searching for additional modules.

thanks. yes that looks like a memory allocation problem. i think 4G is a good size to run vkdt, 2G suffers from my slightly happy memory buffers i keep for fast thumbnail generation. i suppose you might have success running ./vkdt /path/to/image/directly.raw, but even then i might allocate the full thumbnail machinery, just in case. i might need a “low memory” mode, but was hoping i wouldn’t have to do that.