Building Natron on Windows from scratch

eliemichel · May 21, 2018, 2:12pm

/mingw64/lib/pkgconfig/pyside-py2.pc was indeed installed, but the problem came from Shiboken.

$ pkg-config --libs pyside-py2
Package shiboken was not found in the pkg-config search path.
Perhaps you should add the directory containing `shiboken.pc'
to the PKG_CONFIG_PATH environment variable
Package 'shiboken', required by 'pyside-py2', not found

So I did the same for shiboken: pacman -Ql mingw-w64-x86_64-python2-shiboken-qt4. Turns out that the package contains a shiboken-py2.pc but no shiboken.pc and PySide expects the latter. I read tools/MINGW-packages/mingw-w64-pyside-qt4 and I can see in it that any reference to shiboken in PySide package should be rename into shiboken-py2. So I apparently have an inconsistency between my shiboken and pyside packages, one coming from custom repos and not the other one or something like this. I ended up dirty patching my system:

ln -s /mingw64/lib/pkgconfig/shiboken-py2.pc /mingw64/lib/pkgconfig/shiboken.pc

But I’m not happy with it. It’s not something I can release officially in a building How To.

Anyway, I kept on going, and ran qmake. It was not clear which config I should use, since most of them were plotting a warning saying they are designed to be ran at Inria. In particular, I did not manage to set an installation target. Which qmake options would you recommend for an optimized release-like build?

For now I build in the defautl devel mode, and specify an installation path:

qmake -r target.path=/e/SourceCode/Natron/bin

This now runs well, so I go for make then. But at linking time I ran into this issue: Alexpux’ MINGW-packages #1670, with pyside package. But greping my system for some mention to C:\building did not return anything else than the makefiles generated by qmake. So I manually edited those in order to keep going, but again it’s not something I could advise anybody to do.

After that, the make ended correctly. So I ran make install, and checked out my install dir. There were only three elements in it:

Natron.exe
natron-python.exe
NatronRenderer.exe

I copied the libgcc dll it was asking for, but then could not start any of those exe:

pasdemarre

devernay · May 21, 2018, 4:04pm

I would rather edit pyside-py2.pc to make it refer to shiboken-py2 rather than shiboken as a dependency.

As for the other error, ldd is your friend to find out DLL issues. You probably put the wrong libgcc.dll

You can also use Dependency Walker

eliemichel · December 7, 2018, 7:51am

Hello,

I’ve been trying further again to build Natron, and fixed this dll issue, but I’m now facing a new issue: installing PySide. It seems that it is not possible to install PySide on Msys. How is it so? How do you youse to work this around to build Natron for Windows?

É.

abc353940924 · March 24, 2020, 8:55am

Hello, I was also studying natron recently, but I encountered trouble during the compilation process, how did you install the shiboken and pyside packages?

rodlie · March 24, 2020, 6:22pm

We maintain a custom MSYS2 snapshot + repo for Natron, see Natron/tools/MINGW-packages at RB-2.3 · NatronGitHub/Natron · GitHub for more information.

Saku · September 13, 2020, 2:49pm

Hi.

I tried if I could build Natron 2.3 myself on Windows 10, but failed. I have no experience on this and not sure if the end result is installation package, directly running application or something else, but tried to follow instructions to see. I also might very well miss some necessary software if instructions expect prior development experience and knowlege.

But in case someone has time and patience, I’d appreciete help to guide me through this.

I have own fork in Github and synced to my computer. Below are steps that I did and where it failed:

Installed MSYS2 per instructions (Natron/README.md at RB-2.3 · NatronGitHub/Natron · GitHub)
Checked if OpenFX API is up to date (git submodule update -i --recursive)
Created config.pri file with paths to LIBS -folder
Did NOT build nodes (thought maybe not necessary for test)
BUILD FAILED (see below)

Saku@DESKTOP-23R7QK7 MINGW64 /d/PROJECTS/Natron_dev/Development/Natron
$ qmake -r
Reading D:/PROJECTS/Natron_dev/Development/Natron/HostSupport/HostSupport.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/gflags/gflags.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/glog/glog.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/ceres/ceres.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/libmv/libmv.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/openMVG/openMVG.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/qhttpserver/qhttpserver.p ro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/qhttpserver/src/src.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/hoedown/hoedown.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/libtess/libtess.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/Engine/Engine.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/Renderer/Renderer.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/Gui/Gui.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/Tests/Tests.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/PythonBin/PythonBin.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/App/App.pro
Project MESSAGE: You did not select a config option for the build. Defaulting to Devel. You can choose among (snapshot | alpha | beta | RC | stable | custombuil d). For custombuild you need to define the environment variable BUILD_USER_NAME. Also you can give a revision number to the version of Natron with the environme nt variable BUILD_NUMBER (e.g: RC1, RC2 etc…)
Project MESSAGE: Compiling in RELEASE mode.
WARNING: Failure to find: release/Natron_res.o
WARNING: Failure to find: debug/Natron_res.o
WARNING: target.path is not defined: install target not created

WARNING: target.path is not defined: install target not created

WARNING: Failure to find: release/Natron_res.o
WARNING: Failure to find: debug/Natron_res.o
WARNING: target.path is not defined: install target not created

WARNING: target.path is not defined: install target not created

rodlie · September 13, 2020, 7:33pm

You didn’t start the build? type make -jX (X=amount of threads you have)

Saku · September 14, 2020, 6:59am

You are right. I didn’t actually build, but stopped after creating Makefile when seeing warnings, so the last step I mentioned was incorrect.

Now I built it and looked like it finished without errors. Had some warnings though during the process. Now I have no idea where the software was built to?

In Makefile I don’t see any target specified and in my earlier post it had WARNINGS that install target not created. Not sure if there should be a path or not, but here is the beginning of file to see.

first: make_default
MAKEFILE      = Makefile
QMAKE         = c:/msys64-20180531/mingw64/bin/qmake.exe
DEL_FILE      = rm -f
CHK_DIR_EXISTS= test -d
MKDIR         = mkdir -p
COPY          = cp -f
COPY_FILE     = $(COPY)
COPY_DIR      = cp -rf
INSTALL_FILE  = $(COPY_FILE)
INSTALL_PROGRAM = $(COPY_FILE)
INSTALL_DIR   = $(COPY_DIR)
DEL_FILE      = rm -f
SYMLINK       = 
DEL_DIR       = rmdir
MOVE          = mv
CHK_DIR_EXISTS= test -d
MKDIR         = mkdir -p
SUBTARGETS    =  \
		sub-HostSupport \
		sub-libs-gflags \
...
...

rodlie · September 14, 2020, 7:12am

The warnings are normal, I test the SDK regularly, it works

The Natron binary should be in the App folder where you built it.

Saku · September 14, 2020, 8:28am

I see. Unfortunately the /Natron/App/release/ -folder is empty. But maybe I’m not far away to get it working.

/Natron/App/ -folder has Makefile.Debug and Makefile.Release files from yesterday created at the time I made the Makefile and has debug and release folders, but those both folders are empty.

In some other /Natron/“folder” -locations have release folders generated during the build with .o and .cpp files, but I don’t know if that is correct during the building.

/Natron/Gui/release/
/Natron/Renderer/release/
/Natron/Engine/release/
…

I copy here also snippet of the very last part of my build info if that looks like to be correct or if right away can see some problems.

In file included from ../../Natron/Engine/PyParameter.h:41:0,
                 from ../Engine/PyTracker.h:38,
                 from ../Engine/NatronEngine/natronengine_python.h:27,
                 from NatronGui/natrongui_python.h:29,
                 from NatronGui/natrongui_module_wrapper.cpp:13:
../../Natron/Engine/KnobTypes.h:39:1: warning: unknown option after '#pragma GCC diagnostic' kind [-Wpragmas]
 GCC_ONLY_DIAG_OFF(class-memaccess)
 ^~~~~~
g++ -c -include pch/pch.h -pipe -include Python.h -ftemplate-depth-1024 -Wno-deprecated-declarations -fpermissive -Wno-int-in-bool-context -Wno-expansion-to-defined -municode -O2 -g -mms-bitfields -Wall -Wextra -Wall -Wextra -Wno-multichar -Winit-self -Wno-long-long -fexceptions -mthreads -frtti -DUNICODE -DOFX_EXTENSIONS_NUKE -DOFX_EXTENSIONS_TUTTLE -DOFX_EXTENSIONS_VEGAS -DOFX_SUPPORTS_PARAMETRIC -DOFX_EXTENSIONS_TUTTLE -DOFX_EXTENSIONS_NATRON -DOFX_EXTENSIONS_RESOLVE -DOFX_SUPPORTS_OPENGLRENDER -DOFX_SUPPORTS_MULTITHREAD -DOFX_SUPPORTS_DIALOG -DQT_NO_CAST_FROM_ASCII -DNDEBUG -DNATRON_BUILD_NUMBER=0 -DWINDOWS -DCOMPILED_FROM_DSP -DXML_STATIC -DNOMINMAX -D_UNICODE -DUNICODE -DQHTTP_SERVER_STATIC -DQT_DLL -DQT_NO_DEBUG -DQT_NO_KEYWORDS -DQT_OPENGL_LIB -DQT_GUI_LIB -DQT_NETWORK_LIB -DQT_CORE_LIB -DQT_THREAD_SUPPORT -DQT_SHARED -I'c:/msys64-20180531/mingw64/include/qt4/QtCore' -I'c:/msys64-20180531/mingw64/include/qt4/QtNetwork' -I'c:/msys64-20180531/mingw64/include/qt4/QtGui' -I'c:/msys64-20180531/mingw64/include/qt4/QtOpenGL' -I'c:/msys64-20180531/mingw64/include/qt4' -I'../Global/gladRel/include' -I'c:/msys64-20180531/mingw64/include/PySide/QtCore' -I'c:/msys64-20180531/mingw64/include/PySide/QtGui' -I'../libs/OpenFX/include' -I'../libs/OpenFX_extensions' -I'../libs/OpenFX/HostSupport/include' -I'../../Natron' -I'../libs/SequenceParsing' -I'../Engine' -I'../Engine/NatronEngine' -I'../Global' -I'../libs/qhttpserver/src' -I'.' -I'NatronGui' -I'c:/msys64-20180531/mingw64/include/cairo' -I'c:/msys64-20180531/mingw64/include/pixman-1' -I'c:/msys64-20180531/mingw64/include' -I'c:/msys64-20180531/mingw64/include/freetype2' -I'c:/msys64-20180531/mingw64/include/harfbuzz' -I'c:/msys64-20180531/mingw64/include/glib-2.0' -I'c:/msys64-20180531/mingw64/lib/glib-2.0/include' -I'c:/msys64-20180531/mingw64/include/libpng16' -I'c:/msys64-20180531/mingw64/include/python2.7' -I'c:/msys64-20180531/mingw64/include/shiboken' -I'c:/msys64-20180531/mingw64/include/PySide' -I'c:/msys64-20180531/mingw64/include/qt4/ActiveQt' -I'release' -I'c:/msys64-20180531/mingw64/share/qt4/mkspecs/default' -o release/qrc_GuiResources.o release/qrc_GuiResources.cpp
ViewerTab.cpp: In constructor 'Natron::ViewerTab::ViewerTab(const std::__cxx11::list<boost::shared_ptr<Natron::NodeGui> >&, const std::__cxx11::list<boost::shared_ptr<Natron::NodeGui> >&, Natron::Gui*, Natron::ViewerInstance*, QWidget*)':
ViewerTab.cpp:100:1: note: variable tracking size limit exceeded with -fvar-tracking-assignments, retrying without
 ViewerTab::ViewerTab(const std::list<NodeGuiPtr> & existingNodesContext,
 ^~~~~~~~~
ar -M < object_script.libGui.Release
make[2]: Leaving directory '/d/PROJECTS/Natron_dev/Development/Natron/Gui'
make[1]: Leaving directory '/d/PROJECTS/Natron_dev/Development/Natron/Gui'

Saku@DESKTOP-23R7QK7 MINGW64 /d/PROJECTS/Natron_dev/Development/Natron
$

bonalex01 · September 28, 2020, 12:26pm

Hello,
I’m also trying to join the “build Natron on windows wagon”

when i try to clone the sources with GIT i have invalid path problems:
error: invalid path ‘tools/MacPorts/graphics/openimageio/Portfile.’

How do you avoid the problem ?

My understanding is I could add the problematic files to gitignore file but this may be a problem if i wan’t to submit my work on the main Natron branch.

Could you help me ?

rodlie · September 28, 2020, 12:31pm

Not able to replicate, I’m able to clone the source without any problems.

bonalex01 · September 28, 2020, 12:38pm

Thanks, I will look into this.

bonalex01 · September 30, 2020, 3:50pm

Hi, I’d like to keep on trying to make Natron building accessible.
the “git clone natron” fail it was solved by cloning directly from NatronGitHub. My Fork was probably broken.

As Saku did I followed:

github.com

NatronGitHub/Natron/blob/RB-2.3/INSTALL_WINDOWS.md

Instructions for installing Natron from sources on Microsoft Windows
====================================================================

# Installing with MSYS2

[MSYS2](https://sourceforge.net/projects/msys2/) is a unix-like
toolchain that can be used to build Windows applications. It provides
most dependencies required by Natron.

Follow the [Windows SDK](tools/MINGW-packages/README.md) instructions to setup MSYS2 and install the required MSYS packages.

### OpenFX

Natron uses the OpenFX API, before building you should make sure it is up to date.

For that, go under Natron and type

```
git submodule update -i --recursive
```

This file has been truncated. show original

qmake -r
(It worked with many warnings as you mentioned to be normal)

make
It worked but only created 3 files in Natron\App\Release
Natron.exe, Natron_res.o, NatronApp_main.o

When I launch Natron.exe it (not surprinsingly) complains about missing dlls.

During make many messages are of the like:

…/…/Natron/Engine/KnobTypes.h:39:1: warning: unknown option after ‘#pragma GCC diagnostic’ kind [-Wpragmas]
GCC_ONLY_DIAG_OFF(class-memaccess)
^~~~~~

or

c:/msys64-20180531/mingw64/include/qt4/QtCore/qstring.h:1205:69: warning: optimization attribute on ‘bool operator==(const QString&, const QStringRef&)’ follows definition but the attribute doesn’t match [-Wattributes]
Q_CORE_EXPORT bool operator==(const QString &s1,const QStringRef &s2);
^
Could you help us ?

rodlie · September 30, 2020, 8:14pm

I added a tutorial regarding Windows here.

bonalex01 · October 1, 2020, 1:09pm

Thank you so much! I’m probably not the only one who hoped for this for a while.

It’s very clear. (As a newcomer I just needed some time to realize QTCreator was a different install I had to make )

But… I’m still stuck. The result is again a Natron.exe with no dll in the same folder.

During The build QTCreator warned of:
warning: “C:\msys64-20180531\mingw64\bin\gcc.exe” is used by qmake, but “C:\msys64-20180531\mingw64\bin\cc.exe” is configured in the kit.
Please update your kit (Natron) or choose a mkspec for qmake that matches your target environment better.

When I build, I have all these warnings ( I already had the same with the make without QTCreator)
warning: unknown option after ‘#pragma GCC diagnostic’ kind [-Wpragmas]
GCC_ONLY_DIAG_OFF(class-memaccess)

When I launch Natron it just pops the error:
The code execution cannot proceed because libgcc_s_seh-1.dll was not found.
If I copy the dll from somewhere else … of course it asks for an other dll etc…

Maybe I’m not that far…If you still have a little time.

rodlie · October 1, 2020, 1:27pm

If you aim to package Natron then you will need to package it (see the CI environment). The build process will not copy any DLL’s, if you just want to develop Natron then you don’t need to bother with DLL’s as the environment is already setup up (MSYS2/Qt Creator).

bonalex01 · October 1, 2020, 2:11pm

Of course you’re right! My Natron.exe works well if I put it in the folder with all the needed files from a previous install.

Am I supposed to do this manually The build puts the Natron.exe in \App\Release\ folder whereas the normal installer puts it in \bin\ ?
I see in QTCreator some “run settings” that may be the place to ask to copy the Natron.exe to the right place before running it.

michaelquigley · October 2, 2020, 3:26am

Have you tried running Natron.exe from the MSYS shell in your build folder? I run from my development environment like this:

Michael@zero MINGW64 /d/sandbox/natron/App
$ release/Natron.exe

The DLLs that are missing are probably part of MSYS, and will be in your PATH when you run from the MSYS shell.

You’ll also want to get Plugins and Resources from a 2.3.15 release build and put them in your Natron/App folder:

Michael@zero MINGW64 /d/sandbox/natron/App
$ ls -l
total 48
-rw-r--r-- 1 Michael None  2943 Sep 25 11:32 App.pro
drwxr-xr-x 1 Michael None     0 Sep 25 11:56 debug
-rw-r--r-- 1 Michael None  8051 Sep 25 11:56 Makefile
-rw-r--r-- 1 Michael None 10633 Sep 25 11:56 Makefile.Debug
-rw-r--r-- 1 Michael None 10931 Sep 25 11:56 Makefile.Release
-rw-r--r-- 1 Michael None  5263 Sep 25 11:32 NatronApp_main.cpp
-rw-r--r-- 1 Michael None  2165 Sep 25 11:32 NatronInfo.plist.in
drwxr-xr-x 1 Michael None     0 Sep 25 12:02 Plugins
drwxr-xr-x 1 Michael None     0 Sep 27 14:01 release
drwxr-xr-x 1 Michael None     0 Sep 25 12:04 Resources

I also ended up having to add the bin folder from my installed release build of 2.3.15 last in my PATH, to clear up a few other missing DLLs (most image libraries).

Maybe this will help?

rodlie · October 2, 2020, 8:28am

Note that the FFmpeg/libRaw libraries will not be in PATH, as they are in /mingw64/ffmpeg-gpl2/bin and /mingw/libraw-gpl2/bin.

But that’s related to OpenFX-IO and not Natron.