vkdt - some questions

yeah clone+make+happy should be the way to do it.

which img_param did you print there? because the one after the colour module should be identity since it transforms input to rec2020, see colour/main.c:commit_params.
const dt_image_params_t *img_param = dt_module_get_input_img_param(graph, module, dt_token("input")); ← this one should have the camera matrix. module->img_param ← this one holds the metadata for the output.

Thanks for the hint, I assumed that inside create_nodes() would be before applying colour and the matrix would still be intact there?

Anyway, the matrix obtained through dt_module_get_input_img_param(graph, module, dt_token("input")) is correct, and module->img_param.cam_to_rec2020 is identity. So everything ok it seems.

I rewrote commit_params now to avoid the *f and *i offset arithmetic (to rule out different alignments between Linux and macOS/moltenVK at the handover to the shader), using a struct on the C side:
typedef struct {
float mul[4];
float cam_to_rec2020[3][4]; // std140 mat3
uint32_t N[4];
float rbf_P[3][4];
float rbf_c[24][4];
float rbf_p[24][4];
float temp;
uint32_t colour_mode;
float saturation;
uint32_t pick_mode;
uint32_t gamut_mode;
uint32_t primaries;
uint32_t trc;
float clip_highlights;
} colour_params_std140;

But the outcome is as before, RCD fine and gaussian splats colors distorted.

Edit: When I remove all modules after and including colour, I can see that the red channel at the end of demosaic/fix.comp is exactly zero!
Some pixels from the middle grey patch of a color checker image (rgba):
0 0.110413 0.0650024 1
0 0.111633 0.0650635 1
0 0.109863 0.0637817 1

With RCD we get for the same pixels:
0.0499878 0.109619 0.0640869 1
0.0508118 0.110901 0.064209 1
0.052124 0.110352 0.0637207 1

Besides rgb.r, also the corresponding weight w.r is exactly 0 !
wb.g is exactly 1 and wb.b alternates for adjacent pixels between 1.3 and 1.9, all taken with
imageStore(img_out, ipos, vec4(rgb.r, w.r, w.b, 1.0));
gc is ~0,1 .

I will be out for a couple of days from tomorrow on and unable to do more tests, sorry and big thanks for the support so far! Will return latest end of next week.

this is becoming more and more mysterious :slight_smile: you said you installed moltenvk. where does the rest of your sdk come from? on apple silicon/arm macintosh computers i had most success installing the official lunarg vulkan sdk. in particular no brew/other custom moltenvk, it comes with the vulkan sdk.

maybe something in between vk/moltenvk lost track of the single channel textures in the gauss splat code path?

I use homebrew for the more common libraries, and the Vulkan Sdk from here LunarXchange.
Intel iMac 2020, macOS sequoia.

Installation of the libs and make was absolutely flawless, I was really surprised (building darktable took me much more time …).

Actually I have the impression that the red pixel path is never entered in write_bayer2, as I remember having patched that part for testing, e.g. simply assign rgb.r = 1, but with no effect. Unfortunately I am not at my computer to confirm again, will try mid to end of next week.
Btw, I also checked the CFA input to fix.comp and it seemed ok.

I think I found what to change for gaussian splats to work, but I can’t explain it … :frowning:

First, as suspected, the path for the red pixels never seems to be taken in write_bayer2(). If I initialize rgb and the weights in fix.comp/main() like
vec3 rgb = vec3(0.1234, 0.0, 0.0);
vec3 w = vec3(1.0, 0.0, 0.0);
those values make it to the final image output, i.e.
imageStore(img_out, ipos, vec4(rgb, 1.0));
produces
0.123352 0.0178528 0.0114746 1
0.123352 0.0179901 0.0111618 1
0.123352 0.0176697 0.0105896 1
… for subsequent pixels (csv export).

Or
vec3 rgb = vec3(0.5, 0.0, 0.0);
results in
0.5 0.0178528 0.0114746 1
0.5 0.0179901 0.0111618 1
0.5 0.0176697 0.0105896 1
for the same pixels.

Now I also output ipos.x and ipos.y, with following result for that (arbitrary) crop:
2448 1052 0 1
2450 1052 0 1
2450 1052 0 1
2452 1052 0 1
2452 1052 0 1
2454 1052 0 1
2454 1052 0 1
. . .
2576 1052 0 1
2576 1052 0 1
2578 1052 0 1
2578 1052 0 1
2580 1052 0 1
2448 1053 0 1
2450 1053 0 1
2450 1053 0 1
2452 1053 0 1
2452 1053 0 1

I.e. the x coordinate progresses by 2, the y coordinate by 1. Seems strange to me, but might be due to the rggb pattern (or the down-sampled gauss/cov input). Is this ok?

Anyway, per r=1 the parameter “o” to write_bayer2 should clearly also hit red pixels, due to o = ipos+ivec2(i, j) and i, j each taking on -1,0,1 - right?
So I was wondering, why the if(((o.x & 1) == 0) && ((o.y & 1) == 0)) condition should never be met, except … if the green path before always takes precedence and we never hit the … else if(red pixels)

Now the weird thing comes: if I remove the "else"s in the if-r/g/b splitting, the behavior should be unchanged to my understanding, as the if-clauses are mutually exclusive … but actually this activates the red path: gaussian splats now have the correct colors, identical to RCD!
Same if I keep the if-else logic, but just exchange the red and green blocks like
if (red pixels) { … }
else if (green pixels) { … }
else if (blue pixels) { … }
→ Gaussian Splats ok, colors identical to RCD.

Any ideas how this could be explained?

1 Like

oh wow that’s one crazy thing to track down :slight_smile:
maybe something with the driver? just to confirm, you are using a brew-installed moltenvk or the one that comes with the sdk now? i mean i’ll be glad to accept your swapping-the-order patch if that helps to work around it. but really this sounds like some bug in the compiler’s branching logic. branches are of course a bit intricate because they need to split/merge the subgroups.

either patch works for me… leave away else or swap order. sounds a bit discomforting though… we can just wait for this thing to resurface in another compilation unit.

do you want to prepare a pull request or should i just reproduce?

I am using the one from the sdk, not brew-installed.

I would also think that is a compiler bug, maybe some agressive optimization that spoils the logic. But the conditions are actually quite simple, any decent compiler should be able to handle this without side effects.
I also find it discomforting - never sure if such an issue pops up somewhere else, where it is not so easily detectable visually …

BTW, I also changed config.mk.defaults.osx to glslc from glslangValidator and to vulkan 1.4 from 1.3 (as I have installed 1.4.335), but this does not make any difference.

Please just reproduce on your side (suggest to swap order and add a comment as to why the order seems to matter), no need for a PR overhead I think … thanks!

1 Like

pushed, thanks. let me know if this is still broken for you… i’m blindly swapping lines of code here :slight_smile:

It works - thanks :slight_smile:

(Still wondering … )

Unrelated question: until recently, it was possible to zoom out more in darkroom, i.e. have a grey background frame around the image as large as one would like. Now the display seems confined to the window size plus a minimal grey frame around - I can zoom in, but not more out. Is this on purpose?

ah, i noticed that too. you mean via mouse wheel? this is since phcreery changed the mouse wheel to zoom to a specific point under the cursor, not the screen center. i think more freedom is probably better, we can unclamp this (middle mouse will still zoom to fit exactly).

Yes, I meant the mouse wheel.

Changing min_scale to 0.5 is easy enough and works fine - thanks for the hint to that PR.

If I might add a feature wish - adding a color assessment frame like in darktable would be a great enhancement :wink: I fear I am not able to prepare that myself, as I have no understanding of vkdt’s gui part (yet).

… until then you can probably put the frame module in the graph temporarily? maybe with a preset so it’s less annoying.

pushed the unclamp thing.