vkdt on Windows

Software vkdt
21

@rvietor @hanatos Actually some of the linux VFS is emulated by mingw libraries. You can look at my changes but the main issues were:

  • POSIX functions not implemented (or implemented incorrectly) in mingw (e.g. scandir, mkdir, realpath).
  • vkdt code accesses dirent fields directly to check file/directory type. There are many uses of “if(dp->d_type == DT_DIR)” in the code. This is incorrect even on Linux as not all file systems implement it (see readdir(3) - Linux manual page).
  • executable directory was obtained in a non-portable way. I had to work around that.
  • BIG one. vkdt uses directory with symlinks to implement tagging. This is a bad idea even on linux. Not all file systems support symlinks (e.g. user could have photos on an external USB drive formatted as FAT). Also, users would have to be careful copying directory structure or they will end up with copies instead of links. In my experience, emulating database functionality using filesystems is bad design altogether. Of course windows has poor support of links, so I just left tagging broken on windows.
  • In many places vkdt code uses code such as " 1ul << 48". That’s bad style and incorrect, because long is 64 bit on linux, but 32 bit on windows. For this purpose “uint64_t” should be used for portability.
4 Likes
22

A bit off-topic, but is there any reason not to use the int64_t and related types whenever you use an integer type, exactly to avoid such compiler/OS differences? Of course, typing int32_t is more work than just plain int …

23

windows do have hard and soft links, but they are quite hidden, you cannot create or distinguish them in the interface, but there are commands and programming calls to do that (at least in NTFS filesystems, not FAT).

I don’t like symlinks too much as they cause problems when copying files from one place to another and when making and restoring backups (you end up with several copies of files most of the time).

There are POSIX C library in windows that uses the ‘/’ for directory separation but may be it is quite limeted in functionality for the vkdt needs, but for that functionality implemented in the standard C library that provide enough functionality, I think it might be simpler to use them instead of other functions.

If mingw has some problems, have you tried cygwin?
It won’t be not so tiny and needs installation of software in the target platform… but if it can solve the problems…

Cygwin vs Mingw | Top Differences of Cygwin vs Mingw (educba.com)

Differences in integers lenght and endianess, floating point and the likes, has allways been a headache.

My appoach is use a set of typedefs in a .h file to define a set of types like uInteger and dpReal and use them in all places in the program.

Then you just have to change the typedefs to adapt to the platform.
Something like uint64_t but with a more friendly name.

24

I have installed WSL2 in windows and ubuntu in it in order to see if I can test vkdt

It seems that wsl2 lets you run graphics apps and it has nvidia cuda and directml support if you have appropiate driver which exposes a virtual gpu.

It could be a solution for me to run vkdt and even darktable itself (that is a bit limited in windows).

I have selected ubuntu due to its ability to use propietary drivers that might be needed to access advanced features in the graphics card.

I had found here Instalar paquete graphics:darktable:master / vkdt (opensuse.org) how to add it to ubuntu repositories.

But there is a problem: it is only available for ubuntu 21.04 and above, but under WSL2 you can only install ubuntu 20.04 (las version supported).

So how can I try it?
Is it a way to install ubuntu 21 under wsl2?

Should I try debian? (it seems there are no nvidia cuda drivers for debian).

FOLLOW UP:
Well I could finally install Ubuntu 22.04 in WSL2.
It seems you cannot install them through wsl command (wsl -l -o does nos list ubuntu 22.04 just 20.04) but you can install it from microsoft store.

I could install darktable and vkdt.

Darktable works, but there is no opengl hardware.

vkdt does not run because it does not find the appropiate hardware and vulkan libraries.

But as long as I could understand from MS claims, you can get hardware accelerated GPU access from wsl2 VMs.
You just have to install the latest driver in from the gpu manufactures (nvidia in this case) that support wls and install it in windows.

WSL2 kernel provides a virtual gpu driver to access the gpu.

But I could not get it to work.

WELL, I have investigated a bit more.

It seems that openCL and vulkan are not yet ported to WSL2 and so there is no hardware gpu support to it under the virtutal device driver, they have just ported WDDM and D3D12 exposing the API at the device level in user space in linux, and then they have added CUDA and DirectML libraries to talk to the driver and implement that functionality.

But not openCL, openGL or Vulkan yet on top of DirectX 12 driver, they seem to be working in it and building the openGL and openCL API transforming it to DX calls. Vulkan seems that will take a bit longer.

So you cannot use vkdt or even darktable in WSL2 (no GPU accelerated graphics yet for them).

For now the only option that seems viable is the porting initiated by @Gene

25

I should maybe have been more clear earlier when I said “WSL” I meant WSL2/WSLg… I found as you did that even installing the nvidia driver as directed did not yet help. It did state somewhere that OPENCL was not supported so I knew that but I had not confirmed or checked out anything related to vulkan…seems like the same …not yet supported…

1 Like
26

Yes, openCL are supported via cpu emulated library, that won’t help.

The NVIDIA driver is for CUDA, for installing CUDA in linux, it seems.

But once MS implements opengl, opencl or vulkan in WSL2 (yet in development it seems) you won’t need any driver in linux.

You would only update wsl and update your graphic card driver in windows with one that supports wsl and new functionality.

The wsl linux kernel would detect the capabilities and provides de virtual device driver and the libraries to implement the opencl, vulkan and other funcitionality.

You cannot install a driver in linux or the vulkan libraries, as they would override the provided ones for ones that woul try to access a physical card.

DirectX :heart: Linux - DirectX Developer Blog (microsoft.com)
In the works: OpenCL™ and OpenGL® mapping layers to DirectX - DirectX Developer Blog (microsoft.com)

27

Thanks for the link…this is the one I used when I first started to mess around with it…

1 Like
28

Yes they seem to be developing a library in colaboration with Collabora (developers of mesa drivers of directx in linux) that will translate opengl api calls into directx12 calls to the virtualize device driver directx12 capable that wsl kernel provides.

That library will be provided with wsl when available, but I think it is not yet available.

May be it is under wsl insider (beta program).

I have installed mesa-utils and run glxinfo to see opengl capabilities.

It seems that the system has opengl 3.3 capabilities and my nvidia card is detected and available.

glxgears (mesa opengl test runs correctly).

This is the ouput of glxinfo:

name of display: :0
display: :0  screen: 0
direct rendering: Yes
server glx vendor string: SGI
server glx version string: 1.4
server glx extensions:
    GLX_ARB_context_flush_control, GLX_ARB_create_context,
    GLX_ARB_create_context_no_error, GLX_ARB_create_context_profile,
    GLX_ARB_fbconfig_float, GLX_ARB_framebuffer_sRGB, GLX_ARB_multisample,
    GLX_EXT_create_context_es2_profile, GLX_EXT_create_context_es_profile,
    GLX_EXT_fbconfig_packed_float, GLX_EXT_framebuffer_sRGB,
    GLX_EXT_import_context, GLX_EXT_libglvnd, GLX_EXT_no_config_context,
    GLX_EXT_texture_from_pixmap, GLX_EXT_visual_info, GLX_EXT_visual_rating,
    GLX_MESA_copy_sub_buffer, GLX_OML_swap_method, GLX_SGIS_multisample,
    GLX_SGIX_fbconfig, GLX_SGIX_pbuffer, GLX_SGIX_visual_select_group,
    GLX_SGI_make_current_read
client glx vendor string: Mesa Project and SGI
client glx version string: 1.4
client glx extensions:
    GLX_ARB_context_flush_control, GLX_ARB_create_context,
    GLX_ARB_create_context_no_error, GLX_ARB_create_context_profile,
    GLX_ARB_create_context_robustness, GLX_ARB_fbconfig_float,
    GLX_ARB_framebuffer_sRGB, GLX_ARB_get_proc_address, GLX_ARB_multisample,
    GLX_ATI_pixel_format_float, GLX_EXT_buffer_age,
    GLX_EXT_create_context_es2_profile, GLX_EXT_create_context_es_profile,
    GLX_EXT_fbconfig_packed_float, GLX_EXT_framebuffer_sRGB,
    GLX_EXT_import_context, GLX_EXT_no_config_context, GLX_EXT_swap_control,
    GLX_EXT_swap_control_tear, GLX_EXT_texture_from_pixmap,
    GLX_EXT_visual_info, GLX_EXT_visual_rating, GLX_INTEL_swap_event,
    GLX_MESA_copy_sub_buffer, GLX_MESA_multithread_makecurrent,
    GLX_MESA_query_renderer, GLX_MESA_swap_control, GLX_NV_float_buffer,
    GLX_OML_swap_method, GLX_OML_sync_control, GLX_SGIS_multisample,
    GLX_SGIX_fbconfig, GLX_SGIX_pbuffer, GLX_SGIX_visual_select_group,
    GLX_SGI_make_current_read, GLX_SGI_swap_control, GLX_SGI_video_sync
GLX version: 1.4
GLX extensions:
    GLX_ARB_context_flush_control, GLX_ARB_create_context,
    GLX_ARB_create_context_no_error, GLX_ARB_create_context_profile,
    GLX_ARB_fbconfig_float, GLX_ARB_framebuffer_sRGB,
    GLX_ARB_get_proc_address, GLX_ARB_multisample,
    GLX_EXT_create_context_es2_profile, GLX_EXT_create_context_es_profile,
    GLX_EXT_fbconfig_packed_float, GLX_EXT_framebuffer_sRGB,
    GLX_EXT_import_context, GLX_EXT_no_config_context,
    GLX_EXT_texture_from_pixmap, GLX_EXT_visual_info, GLX_EXT_visual_rating,
    GLX_MESA_copy_sub_buffer, GLX_MESA_query_renderer, GLX_OML_swap_method,
    GLX_SGIS_multisample, GLX_SGIX_fbconfig, GLX_SGIX_pbuffer,
    GLX_SGIX_visual_select_group, GLX_SGI_make_current_read
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: Microsoft Corporation (0xffffffff)
    Device: D3D12 (NVIDIA GeForce GTX 1050 Ti) (0xffffffff)
    Version: 22.0.1
    Accelerated: yes
    Video memory: 12143MB
    Unified memory: no
    Preferred profile: core (0x1)
    Max core profile version: 3.3
    Max compat profile version: 3.3
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.1
OpenGL vendor string: Microsoft Corporation
OpenGL renderer string: D3D12 (NVIDIA GeForce GTX 1050 Ti)
OpenGL core profile version string: 3.3 (Core Profile) Mesa 22.0.1
OpenGL core profile shading language version string: 3.30
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
    GL_AMD_conservative_depth, GL_AMD_draw_buffers_blend,
    GL_AMD_multi_draw_indirect, GL_AMD_shader_trinary_minmax,
    GL_AMD_texture_texture4, GL_ANGLE_texture_compression_dxt3,
    GL_ANGLE_texture_compression_dxt5, GL_ARB_ES2_compatibility,
    GL_ARB_ES3_1_compatibility, GL_ARB_ES3_compatibility,
    GL_ARB_arrays_of_arrays, GL_ARB_base_instance, GL_ARB_blend_func_extended,
    GL_ARB_clear_buffer_object, GL_ARB_compressed_texture_pixel_storage,
    GL_ARB_compute_shader, GL_ARB_compute_variable_group_size,
    GL_ARB_conditional_render_inverted, GL_ARB_conservative_depth,
    GL_ARB_copy_buffer, GL_ARB_debug_output, GL_ARB_depth_buffer_float,
    GL_ARB_depth_clamp, GL_ARB_direct_state_access, GL_ARB_draw_buffers,
    GL_ARB_draw_buffers_blend, GL_ARB_draw_elements_base_vertex,
    GL_ARB_draw_indirect, GL_ARB_draw_instanced,
    GL_ARB_explicit_attrib_location, GL_ARB_explicit_uniform_location,
    GL_ARB_fragment_coord_conventions, GL_ARB_fragment_shader,
    GL_ARB_framebuffer_no_attachments, GL_ARB_framebuffer_object,
    GL_ARB_framebuffer_sRGB, GL_ARB_get_program_binary,
    GL_ARB_get_texture_sub_image, GL_ARB_gpu_shader5, GL_ARB_half_float_pixel,
    GL_ARB_half_float_vertex, GL_ARB_indirect_parameters,
    GL_ARB_instanced_arrays, GL_ARB_internalformat_query,
    GL_ARB_internalformat_query2, GL_ARB_invalidate_subdata,
    GL_ARB_map_buffer_alignment, GL_ARB_map_buffer_range, GL_ARB_multi_bind,
    GL_ARB_multi_draw_indirect, GL_ARB_occlusion_query2,
    GL_ARB_parallel_shader_compile, GL_ARB_pixel_buffer_object,
    GL_ARB_point_sprite, GL_ARB_program_interface_query,
    GL_ARB_provoking_vertex, GL_ARB_robustness, GL_ARB_sample_shading,
    GL_ARB_sampler_objects, GL_ARB_seamless_cube_map,
    GL_ARB_separate_shader_objects, GL_ARB_shader_atomic_counter_ops,
    GL_ARB_shader_atomic_counters, GL_ARB_shader_bit_encoding,
    GL_ARB_shader_draw_parameters, GL_ARB_shader_image_load_store,
    GL_ARB_shader_image_size, GL_ARB_shader_objects,
    GL_ARB_shader_storage_buffer_object, GL_ARB_shader_subroutine,
    GL_ARB_shader_texture_lod, GL_ARB_shading_language_420pack,
    GL_ARB_shading_language_include, GL_ARB_shading_language_packing,
    GL_ARB_stencil_texturing, GL_ARB_sync, GL_ARB_tessellation_shader,
    GL_ARB_texture_buffer_object, GL_ARB_texture_buffer_object_rgb32,
    GL_ARB_texture_buffer_range, GL_ARB_texture_compression_rgtc,
    GL_ARB_texture_cube_map_array, GL_ARB_texture_filter_anisotropic,
    GL_ARB_texture_float, GL_ARB_texture_gather, GL_ARB_texture_multisample,
    GL_ARB_texture_non_power_of_two, GL_ARB_texture_query_levels,
    GL_ARB_texture_query_lod, GL_ARB_texture_rectangle, GL_ARB_texture_rg,
    GL_ARB_texture_rgb10_a2ui, GL_ARB_texture_stencil8,
    GL_ARB_texture_storage, GL_ARB_texture_storage_multisample,
    GL_ARB_texture_swizzle, GL_ARB_timer_query, GL_ARB_transform_feedback2,
    GL_ARB_transform_feedback3, GL_ARB_transform_feedback_instanced,
    GL_ARB_uniform_buffer_object, GL_ARB_vertex_array_bgra,
    GL_ARB_vertex_array_object, GL_ARB_vertex_attrib_binding,
    GL_ARB_vertex_buffer_object, GL_ARB_vertex_shader,
    GL_ARB_vertex_type_10f_11f_11f_rev, GL_ARB_vertex_type_2_10_10_10_rev,
    GL_ATI_blend_equation_separate, GL_ATI_texture_float,
    GL_EXT_EGL_image_storage, GL_EXT_EGL_sync, GL_EXT_abgr,
    GL_EXT_blend_equation_separate, GL_EXT_draw_buffers2,
    GL_EXT_draw_instanced, GL_EXT_framebuffer_blit,
    GL_EXT_framebuffer_multisample, GL_EXT_framebuffer_multisample_blit_scaled,
    GL_EXT_framebuffer_object, GL_EXT_framebuffer_sRGB,
    GL_EXT_packed_depth_stencil, GL_EXT_packed_float,
    GL_EXT_pixel_buffer_object, GL_EXT_provoking_vertex,
    GL_EXT_shader_integer_mix, GL_EXT_texture_array,
    GL_EXT_texture_compression_dxt1, GL_EXT_texture_compression_rgtc,
    GL_EXT_texture_compression_s3tc, GL_EXT_texture_filter_anisotropic,
    GL_EXT_texture_integer, GL_EXT_texture_sRGB, GL_EXT_texture_sRGB_decode,
    GL_EXT_texture_shared_exponent, GL_EXT_texture_snorm,
    GL_EXT_texture_swizzle, GL_EXT_timer_query, GL_EXT_transform_feedback,
    GL_EXT_vertex_array_bgra, GL_IBM_multimode_draw_arrays,
    GL_KHR_context_flush_control, GL_KHR_debug, GL_KHR_no_error,
    GL_KHR_parallel_shader_compile, GL_KHR_texture_compression_astc_ldr,
    GL_KHR_texture_compression_astc_sliced_3d, GL_MESA_pack_invert,
    GL_MESA_shader_integer_functions, GL_MESA_texture_signed_rgba,
    GL_NV_conditional_render, GL_NV_copy_image, GL_NV_depth_clamp,
    GL_NV_packed_depth_stencil, GL_OES_EGL_image, GL_S3_s3tc

OpenGL version string: 3.3 (Compatibility Profile) Mesa 22.0.1
OpenGL shading language version string: 3.30
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile
OpenGL extensions:
    GL_AMD_conservative_depth, GL_AMD_draw_buffers_blend,
    GL_AMD_multi_draw_indirect, GL_AMD_shader_trinary_minmax,
    GL_AMD_texture_texture4, GL_ANGLE_texture_compression_dxt3,
    GL_ANGLE_texture_compression_dxt5, GL_APPLE_packed_pixels,
    GL_ARB_ES2_compatibility, GL_ARB_ES3_1_compatibility,
    GL_ARB_ES3_compatibility, GL_ARB_arrays_of_arrays, GL_ARB_base_instance,
    GL_ARB_blend_func_extended, GL_ARB_clear_buffer_object,
    GL_ARB_color_buffer_float, GL_ARB_compatibility,
    GL_ARB_compressed_texture_pixel_storage, GL_ARB_compute_shader,
    GL_ARB_compute_variable_group_size, GL_ARB_conditional_render_inverted,
    GL_ARB_conservative_depth, GL_ARB_copy_buffer, GL_ARB_debug_output,
    GL_ARB_depth_buffer_float, GL_ARB_depth_clamp, GL_ARB_depth_texture,
    GL_ARB_direct_state_access, GL_ARB_draw_buffers,
    GL_ARB_draw_buffers_blend, GL_ARB_draw_elements_base_vertex,
    GL_ARB_draw_indirect, GL_ARB_draw_instanced,
    GL_ARB_explicit_attrib_location, GL_ARB_explicit_uniform_location,
    GL_ARB_fragment_coord_conventions, GL_ARB_fragment_program,
    GL_ARB_fragment_program_shadow, GL_ARB_fragment_shader,
    GL_ARB_framebuffer_no_attachments, GL_ARB_framebuffer_object,
    GL_ARB_framebuffer_sRGB, GL_ARB_get_program_binary,
    GL_ARB_get_texture_sub_image, GL_ARB_gpu_shader5, GL_ARB_half_float_pixel,
    GL_ARB_half_float_vertex, GL_ARB_indirect_parameters,
    GL_ARB_instanced_arrays, GL_ARB_internalformat_query,
    GL_ARB_internalformat_query2, GL_ARB_invalidate_subdata,
    GL_ARB_map_buffer_alignment, GL_ARB_map_buffer_range, GL_ARB_multi_bind,
    GL_ARB_multi_draw_indirect, GL_ARB_multisample, GL_ARB_multitexture,
    GL_ARB_occlusion_query, GL_ARB_occlusion_query2,
    GL_ARB_parallel_shader_compile, GL_ARB_pixel_buffer_object,
    GL_ARB_point_parameters, GL_ARB_point_sprite,
    GL_ARB_program_interface_query, GL_ARB_provoking_vertex,
    GL_ARB_robustness, GL_ARB_sample_shading, GL_ARB_sampler_objects,
    GL_ARB_seamless_cube_map, GL_ARB_separate_shader_objects,

.....  continues ...

So it seems that hardware opengl support is already there.

But opencl seems not to work, I have installed clinfo and it returns 0 platforms.

Do you have to install a separate opencl package in linux?

29

This thread is about vkdt, let’s get back on subject please.

1 Like
30

I agree. Split to a new thread. Enjoy!

31

Not too much to be said. We were talking about running It on Windows, not everybody uses Linux.

But It does not work on wsl for now, may be in the future when vulkan si finally implementes un WSL.

For Now let’s put the hope on @Gene convertion work.

Great to have a thread about vdkt un Windows, thanks.

32

Gerrit NoteDB :grinning:

33

But isn’t that the principle of flat file cms as well? I think it does have advantages

34

So does it work on Windows? I see no binary package.

35

I think it depends on where you draw the border regarding “database functionality”. As often in life, reality is not as easy to put into categories as it is to invent these categories.

Pulling a couple of html documents from the harddisk into ram is hardly something that benefits from a “real” database system, but running services such as wikipedia or amazon.com will hardly be possible without.

Eventually, a file system is anyway a database for files …

2 Likes
36

agree. that was the idea: database people laughed at me for using a full blown sql backend for trivial things as required in the original darktable (no multi-user! no network connection! barely a million images in it!). in vkdt i wanted even less. some minimal tagging, but mostly low-entry-bar display of raw files which are already organised in folders. in particular i didn’t want to import/mirror the structure in some hidden layer that might be corrupted during debugging or schema updates.

3 Likes
37

this one is interesting. (it’s just too easy to make fun of an operating system that thinks today 32-bit wide registers are particularly “long” so i will refrain from that. iirc linux started to call 64 bit l instead of ll once the pointer type changed to that, i.e. on 64-bit systems). it appears as for everything, there is a portable way to do such literals for typed constants. of course there’s a whole customisable layer of bloat in c++. but even in c, in stdint.h we can find macros that will append the platform specific suffix for us. for instance i have:

# define UINT8_C(c) c
# define UINT16_C(c)  c
# define UINT32_C(c)  c ## U
# if __WORDSIZE == 64
#  define UINT64_C(c) c ## UL
# else
#  define UINT64_C(c) c ## ULL
# endif

which appends the correct letters. unfortunately that takes me from an innocent 1ul to UINT64_C(1) which is not much shorter or better to read than ((uint64_t)1). suppose i can define ul1 to something, but either way this is ugly.

38

Windows 11 WSL has graphics and acceleration support right? I never tried but I was thinking OpenCL should be possible in theory…

39

It’s not about the OS, the the compiler :wink: . And most importantly, the need / wish to stay compatible with all the legacy.

The Windows API and other compilers decided to keep the meaning as it was in the 32bit era to keep older code source-compatible.

Working on a project for years with a Windows 32bit executable / version and a Linux (ubuntu 9 / 10) version… argh, the horrors (but the lessons learned…) in coding, so it compiles on both environments and actually works :wink: .

1 Like
40

No Opencl yet but I haven’t followed for a couple of months…it can look like it does but I think no …only CUDA support