vkdt guide on apple silicon/intel

Software vkdt
21

It looks like the new build is linked to homebrew paths and not the internal ones in /Frameworks

“Library not loaded: /opt/homebrew/*/libglfw.3.dylib”

Modifying it to use the internal libgflw.3.dylib fixes it and everything else seems to work normally:

bash -lc '
APP="/Applications/vkdt.app"
BIN="$APP/Contents/MacOS/lib/vkdt/vkdt"

echo "Before:"
otool -L "$BIN" | grep -i glfw || true

# find the current glfw reference
OLD="$(otool -L "$BIN" | awk "/libglfw/{print \$1; exit}")"
echo "Old ref: $OLD"

# change it to a bundle-friendly reference
sudo install_name_tool -change "$OLD" "@rpath/libglfw.3.dylib" "$BIN"

# ensure the executable searches the app bundle Frameworks
sudo install_name_tool -add_rpath "@executable_path/../../Frameworks" "$BIN" 2>/dev/null || true

echo "After:"
otool -L "$BIN" | grep -i glfw || true

# re-sign
sudo codesign --force --deep --sign - "$APP"
sudo xattr -dr com.apple.quarantine "$APP"
'
22

uhm shouldn’t this be why we have

echo "OPT_LDFLAGS=\"-Wl,-rpath,@executable_path/../../../Frameworks\"" > bin/config.mk

earlier in the ci script? does that not work? glfw is the only dynamically linked dependency of the vkdt binary. your replacement seems to have one ../ less in the path to the Frameworks. might that be the issue?

1 Like
23

…I did have one less ../ in my snippet. Not totally sure why it still worked on my machine! But agreed the correct path from Contents/MacOS/lib/vkdt/vkdt to Contents/Frameworks is @executable_path/../../../Frameworks

However, testing with otool -L shows the vkdt binary is linked to an absolute Homebrew install name for glfw:

/opt/homebrew/opt/glfw/lib/libglfw.3.dylib

Even though Contents/Frameworks/libglfw.3.dylib exists, otool -D on it reports its ID is still /opt/homebrew/opt/glfw/lib/libglfw.3.dylib (so it’s been bundled but not rewritten to an @rpath/...).

EDIT:

Looks like it might be similar for Vulkan: otool -L on vkdt shows it’s linked to /opt/homebrew/opt/vulkan-loader/lib/libvulkan.1.dylib

24

… would this also apply to modules that link external libraries, such as ffmpeg for i-vid or jpg for i-jpg? also probably the command line binary is affected :expressionless:

1 Like
25

…Yep :face_in_clouds: Looks like any module that uses external/Homebrew dylibs are still referencing absolute /opt/homebrew/... paths instead of @rpath/...

26

Yes this applies to all the homebrew dylibs.

If I recall correctly — all the homebrew dylibs are built with their LC_ID_DYLIB value set to their absolute path, where typically I think dylibs are expected to be built with LC_ID_DYLIB set to @rpath/NAME_OF_DYLIB. Thus when the vkdt executables are built they depend on the absolute path, which doesn’t contain any @rpath that dyld can substitute.

I’ve encountered several discussions where people request they switch, but I believe homebrew’s stance is that this is partially deliberate, to discourage people bundling their binaries in other projects.

I think I solved this with a hideous line in my script:

find ~/vkdt.app -type f -print0 | xargs -0 file | grep Mach-O | cut -wf 1 | sed 's/.$//' | tr '\n' '\0' | xargs -0 -n1 sh -c 'otool -L $0 | tail -n +2 | grep -v "/System/" | grep -v "/usr/lib" | grep -v "@rpath" | cut -wf 2 | xargs echo $0' | tr '\n' '\0' | xargs -0 -n1 bash -c 'depender=$(echo $0 | cut -wf 1); dependent=${0//$depender /}; for i in $dependent; do install_name_tool -change $i @rpath/$(basename $i) $depender; done'

But I’m sure that could be written better.

1 Like
27

awesome, love it :smiley:

from what i gather from random webpages, the install name of a dylib is fixed at library compile time and needs to be replaced by hideous means after linking anyways, so we might as well…

28

“ci: macintosh dylib path madness” :laughing:

Good news is It looks like referencing is fixed! It’s using @rpath now. However libglfw.3.dylib is not present in @frameworks :slight_smile:

29

uhm… next try is online. i did write the new script on a macintosh lapop… only it’s one from 10+ years ago and an intel. it does run vkdt when i build it there, but it doesn’t run the dmg that the github ci provides, i assume it’s only good for arm/apple silicon.

1 Like
30

And it appears to work! Thanks so much for doing all this :grinning:

Excited to try everything out properly over the coming weeks

31

The latest Github image again launches on my computer, but not on a VM. But I can’t find any reference to a homebrew path in any linked dylib. I have a vague suspicion it might be something to do with the Vulkan Loader and MoltenVK, and/or more-nuanced dylib LC_ID_DYLIB issues, but I don’t really know and will keep looking.

(Also for reference I have largely rewritten my build script to use hopefully less horrendous bash, but it currently doesn’t produce a working app.)

By the way, the GitHub workflow runs on ‘macos-14’ which is an arm64 machine, but Intel ones are available too.

1 Like
32

great, thanks for checking. i might set up a second instance with intel then. will look at your script to check differences.

the current ci script only explicitly repoints brew libraries, but there are references outside this too. mostly looks like core system stuff to me, but there is also libstdc++ (for mcraw support) and libz (used in various places like exr).

as to vulkan: the dmg only ships libvulkan.so (or what’s it called on macintosh), which depends on the vendor-specific backends to be present elsewhere.

33

I was meaning that in the GitHub bundle there weren’t any remaining homebrew load paths, thus the problem is presumably elsewhere.

On Vulkan, does that mean you don’t intend to bundle MoltenVK? libMoltenVK.dylib is currently in the GitHub bundle due to a special case. It isn’t referenced by an LC_LOAD_PATH and is instead located by the Vulkan Loader (which I assume is included in libvulkan) based on the path in MoltenVK_icd.json. Or at least I think that’s how it’s meant to work. At the moment it isn’t, and is falling back to homebrew libraries if installed, or nothing and crashing if not.

34

okay. i think the way to do it is to remove the references to moltenvk (it’s included with newer vk sdks) and use the brew vulkan sdk for the ci build. there are vulkan sdk actions for github, but the caveat is that these pull in gigabytes of dependencies and the runners don’t have a gpu to actually test it. shipping gigabytes of shaderc static libraries in a dmg doesn’t sound like a good idea to me. ideally the (shipped) vulkan loader would walk the host system’s ICD files and find any installed sdk (brew or otherwise) during run time.

1 Like
35

Hmmm, I won’t claim to have a good understanding here (and I don’t even have a background in programming) so my apologies if I am making some obvious oversights.

But for instance, I thought the current build already exclusively uses homebrew provided vulkan libraries? (Including that MoltenVK library?)

And also, I thought that MoltenVK was a (relatively) thin layer that uhhh ‘translates’ Vulkan to Metal. And if I recall correctly in previous attempts I could get a working vkdt app on a VM, without homebrew or the VulkanSDK installed, by bundling libMoltenVK.dylib. (libMoltenVK is ~5.2 MB, much larger than libvulkan at ~650 KB but smaller than say libavcodec at ~9.7 MB).

If that’s possible with only libMoltenVK the size penalty seems reasonable, and it would importantly allow vkdt to be used without the user having to install any additional software.

2 Likes
36

I may have got excited too quickly! libvulkan.1.dylib and libMoltenVK.dylib were present in Homebrew, and removing those made VKDT unable to load.

I’m a bit out of my depth here, but will come back with any findings.

37

I have got a bundled MoltenVK to work again (and reading the vulkan documentation it does seem this is the recommended route). It is mentioned here that when in a bundle, the Vulkan loader will search for a Driver Manifest in <bundle>/Contents/Resources/vulkan/icd.d, which then points the loader to the MoltenVK library. If it finds a driver in the bundle, this will take precedence over all others.

Setting VK_LOADER_DEBUG=all allows you to see the paths searched for the manifest, and instead of searching <bundle>/Contents/Resources/vulkan/icd.d it kept searching in <bundle>/Contents/MacOS/lib/vkdt/vulkan/icd.d. I still don’t know why! But if I move the main executable of the bundle from <bundle>/Contents/MacOS/lib/vkdt/vkdt to <bundle>/Contents/MacOS/vkdt (as is probably more typical), the loader will correctly search for the manifest in the Resources folder and the app now launches for me on a fresh VM.
Though if homebrew MoltenVK is installed the bundled MoltenVK doesn’t take precedence. For some reason. Concerningly this is accompanied by the following line in vkdt -d output:

objc[27024]: Class MVKBlockObserver is implemented in both /Volumes/vkdt Image/vkdt.app/Contents/Frameworks/libMoltenVK.dylib (0x113c67120) and /opt/homebrew/Cellar/molten-vk/1.4.1/lib/libMoltenVK.dylib (0x11415a6a8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.

Also, and the following might not matter as the Github workflow differs, but the app my script makes shows no thumbnails and won’t load load images :frowning:
Full vkdt -d output is:

[gui] vkdt 1.0.99-210-g7a9e8295 (c) 2020--2026 johannes hanika
[gui] glfwGetVersionString() : 3.4.0 Cocoa NSGL Null EGL OSMesa monotonic dynamic
objc[27024]: Class MVKBlockObserver is implemented in both /Volumes/vkdt Image/vkdt.app/Contents/Frameworks/libMoltenVK.dylib (0x113c67120) and /opt/homebrew/Cellar/molten-vk/1.4.1/lib/libMoltenVK.dylib (0x11415a6a8). This may cause spurious casting failures and mysterious crashes. One of the duplicates must be removed or renamed.
[gui] monitor [0] Built-in Retina Display at 0 0
[gui] vk extension required by GLFW:
[gui]   VK_KHR_surface
[gui]   VK_EXT_metal_surface
[gui] no gamepad found
[ERR] module thumb main connector input has uninited size!
[ERR] [thm] failed to run first half of graph!
[ERR] could not load required thumbnail symbols!

This shouldn’t be to do with sandboxing, as it’s not enabled. And besides the app uses the scale size set in my config.rc and the fonts I put in ~/.config/vkdt/data. So I don’t really know where to look.

Here is the app and my current script (with some comments) for reference.

38

nice!

did you move everything from lib/vkdt up two levels? the resource search path is relative to the location of the executable. if you move only vkdt in isolation it will not find stuff (for instance default graphs for thumbnails or the busy loading icons etc).

39

My bundle includes a launcher script that exports the MoltenVK_icd.json location and then invokes vkdt (in its original location, like indicated by @hanatos).
Here’s the code from my script to create the bundle:

MACOS="$APPDIR/Contents/MacOS"

cat > "$MACOS/vkdt" <<'EOF'
#!/bin/zsh
APPROOT="$(cd "$(dirname "$0")/../.." && pwd)"
export VK_ICD_FILENAMES="$APPROOT/Contents/Resources/vulkan/icd.d/MoltenVK_icd.json"
cd "$APPROOT/Contents/MacOS/lib/vkdt"
exec ./vkdt "$@"
EOF

chmod +x "$MACOS/vkdt"

I also had that “could not load thumbnail symbols” error, in my case it was caused by vkdt not finding other libraries, unrelated to vulkan.

40

I did:

mv "$APPDIR"/Contents/MacOS/lib/vkdt/* "$APPDIR"/Contents/MacOS/
rmdir "$APPDIR"/Contents/MacOS/lib/vkdt
rmdir "$APPDIR"/Contents/MacOS/lib

I partially solved it by removing a line that moved all the the dylibs in the bundle to Contents/Frameworks as mentioned earlier in the thread (I can’t quite remember why, but I was at some point under the impression that all dylibs had to be in Contents/Frameworks to make codesign happy, but I suppose not) Now thumbnails load, but I can’t open images.