Developing and Fiddling with the G'MIC Python Binding

Yes, I will want to use it “headless”. :slight_smile:

Well, I don’t have a Mac and have actually never used one either…

I am now looking for cross-platform support because someone else was kind enough to volunteer to work on getting Gimel Studio up-to-speed on MacOs. Thus, Gimel Studio can be cross-platform. The port has been successful and quite smooth, with only a few minor issues yet to work out, thankfully… :smiley:

(Trying to do in “in the dark”, headless, with Github Actions, as you are doing with windows and mac here, is very difficult! Trial-and-error is a rough road…)

It looks like someone else tested it on Mac though, which is good. Though, if I understood correctly, it doesn’t seem to be working. :frowning:

o.k. I am patiently waiting for this… :wink:

Alright! :slightly_smiling_face:

1 Like

@KaRo
Thank you for listing the libraries dependencies for the gmic-py .so and the gmic executable and the appropriate relinking tool on MacOS. I was about to ask you for it but you did it for me thanks!

And so once you add that flag, does the executable command line run “gmic sp display” pop up a display window?

Thanks for your clarity. It is nice that someone has picked up MacOS build for Gimel Studio.

There are obviously no code changes needed anymore to gmic-py/libgmic for MacOS support, but a build system that would produce a gmic-py working for any MacOS user without anything or too much things to install additionnally is indeed not landed yet.

really nice!!

I found that something has changed, perhaps in the latest Makefile changes, in building gmic cli, SUBLIBS=-lX11 is not necessary anymore, prerelease version!

Seemingly in your build system “x11” is not found, since in my case I got “Library not found”. Your build is clearly made without display.

Using you build tool I need info how to chose the appropriate compiler, still it was a long searching to find adequate compile options. Actually my gmic cli build command is:
(cd ~/work/src/gmic/src/; make clean; make -B CXX=clang++ OPT_CLI_CFLAGS="-march=native -O3 -flto" clean cli || exit; sudo cp ~/work/src/gmic/src/gmic /usr/local/bin/)

Never tried to build libgmic!

OK…
the setup.py build script strategy for gmic-py leverages pkg-config on UNIX-like OSes, in order to enable cflags (includes…) and linkable libraries.

On MacOS, if both the pkg-config executable is installed, and it knows how to generate x11 cflags and include parameters for the compiler, then x11 should be linked against. This is what those setup.py lines seem to imply, per reading my old code :slightly_smiling_face:

Possibly, the vanilla Github Actions compiling environment has no trace of X11 (library+xorg server or X11 app), even though one could install it through the command line after installing MacPorts… hence no automatic X11 detection through pkg-config.

You seem to insist wanting to try to compile gmic-py, which me be a big step for humanity. I think that I have been the only one doing it for now. If so, read next.
Per this page on compiling gmic-py, you can try grabbing the gmic-py 2.8.4 cross-platform source tarball and compile it on the fly with the following command (at cleanest and easiest in a python 3.6/3.7 virtual environment):
pip install gmic --compile
and keep iterating until dependencies are solved. You will want to pip uninstall gmic -y first.
Dependencies are:

  • (brew install) pkg-config fftw libpng libomp llvm@6 coreutils
  • X11 (never tried - but hopefully a X11 that either can be detected by pkg-config, otherwise I/you could add it as an additional compile flag in the setup.py)

Newer gmic-py versions will want libjpeg and libtiff, but not 2.8.4.

Tell me if you would like to build from a Git clone and be able to finetune the setup.py the build script for MacOS, if ever you have time and motivation for this… I can otherwise send setup.py MacOS-related variants to Github for you to pull and test recompiling.

setup.py builds gmic-py and libgmic all at once, the former completely embedding the latter, without any pre-compiled static library.

Even if the current llvm you have is not version 6, compiling should work, with possibly OpenMP not enabled yet… This is an OK step for troubleshooting X display instead.

I think I can try to compile from git

What I did:

cd ~/work/src
git clone …
export GMIC_VERSION=prerelease
cd gmic-py

bash build_tools.bash 1_clean_and_regrab_gmic_src

bash build_tools.bash 2_compile
.
.
.
/usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk -Dgmic_build -Dcimg_date="" -Dcimg_time="" -Dgmic_is_parallel -Dcimg_use_zlib -Dcimg_use_openmp -Dgmic_py_numpy -Dgmic_py_jupyter_ipython_display -Dcimg_display -Dcimg_use_png -Dcimg_use_tiff -Dcimg_use_jpeg -Dcimg_use_fftw3 -Dcimg_use_opencv -I/opt/local/include -I/opt/local/include/libpng16 -I/opt/local/include -I/opt/local/include/opencv -I/opt/local/include -I/Volumes/mbkaroSD/Users/karo/sw/gmic-py -I/Volumes/mbkaroSD/Users/karo/sw/gmic-py/src/gmic/src -I/usr/local/opt/llvm@6/include -I/Users/karo/env/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c gmicpy.cpp -o build/temp.macosx-10.14-x86_64-3.6/gmicpy.o -std=c++11 -Ofast -flto -fopenmp -stdlib=libc++
clang: error: unsupported option ‘-fopenmp’
error: command ‘/usr/bin/clang’ failed with exit status 1
Fatal setup.py build error

How to change the compiler? Of course, there is no /usr/local/opt/llvm@6 in my environment!
And how to get rid of openmp ?

Thank you very much for your steps in the very right direction!!!

For info, I have not tested compilation of gmic-py against libgmic 2.9.5 (prerelease), having a void GMIC_VERSION variable will make the build_tools.bash script look up the git repository’s rootVERSION file’s contents, which is 2.9.4a1 or so, equating to downloading libgmic 2.9.4 but tagging it as 2.9.4 alpha 1 when packaging for Python… Or you can safely export GMIC_VERSION=2.9.4 instead if you prefer.

For switching compiler, try this CC environment variable overriding, through a shell export command as you did for GMIC_VERSION, without patching any part of the setup.py file for now.

For the -fopenmp option, if your other compiler does not support it, you can tune these lines OpenMP related lines in setup.py:

This troubleshooting project shows how OpenMP can be enabled and working for a tiny simple C project using the default brew CLang on Mac OS 10.15 (current macos-latest on Github Actions), this is not a Python project though and not LLVM/CLang:

Thanks for your time helping out!!

If you lack time, you can use the bash build_tools.bash 2b_compile_debug command which has no optimization on (apart from openmp enabled) and builds faster

Ok, just tried to 2_compile:

only changed in setup.py a -L/opt/local/lib/libomp and CC and CXX with clang…-mp-6.0

A bundle is built, how to continue?

go to build/lib/ some build subdirectory
you see a .so or .dynlib file there
once you are in that subdirectory where the shared library lives, pip uninstall -y gmic, then just run python3 which will pick the gmic*.* shared library in the current directory and write gmic-py code like the following in the python interactive console:

import gmic
gmic.run("sp leno blur 3 display")
print(gmic.__version__)
print(gmic.__spec__) # shows which shared library is loaded, so that you ensure your are not loading from an installed wheel file
print(gmic.__build__) # see which libgmic macros were enabled
# use the tutorials 1, 2, 3 or the quickstart tutorial on https://gmic-py.readthedocs.io/en/latest/

you can also run some .py file with the exact kind of contents as above

the next step will be to build a wheel for other people for macos… let us say that I can update the macos Github Actions workflow file after then and release for macos for each new release once we find a proper packaged wheel directory tree with no shared libraries issues as you had

the oneliner to test for openmp really working is the ugly string on this line (just put the string as a parameter in a a gmic.run() call.

and the gmic.run("sp leno blur 3 display") call attempts to open a up a X windows, which is what I am looking after most critically (then help the gmic-py tutorials be easier to follow), next to openmp support

the openmp test string result output must be non zero for a successful test

o.k. Thanks. I hope whatever issues needing to be resolved are resolved for MacOs and Windows. I did try gmic on linux and opened an issue about a small omission from the README in the Github repo. :smiley:

2 Likes

perfect thank you fixed!

1 Like

Just one last

gmic.run(“+v up”)

gmic: GREYC’s Magic for Image Computing: command-line interface
Version 2.9.5
(https://gmic.eu)

Copyright (c) 2008-2021, David Tschumperlé / GREYC / CNRS.
(https://www.greyc.fr)

however version shows 2.9.4!

gmic.__version__ is the packaging version of the python wrapper, which comes from the VERSION file at the git cloned directory’s root, it is just decorative as long as you do not build and share python wheel files to other people who will believe in this version. This can be different from the libgmic version you have contained in the src/gmic/src/gmic.h file, because as a devops guy you build whatever wrapper version with whatever libgmic as you will.

gmic.run("+v up") is the G’MIC version of the libgmic runtime that is embedded, this is the reality of what your gmic-py shared library / wheel archive contains

How are the openmp enabled and display window tests going?