Building Natron on Windows from scratch

Hi! I am trying to build Natron from scratch and run into a couple of interrogations.

First of all, I had to figure out which repository I should use, because natron.fr points to MrKepzie’s repo while the official one actually is NatronGitHub/Natron.

I am building on Windows (with MSYS2), and so logically had a look at INSTALL_WINDOWS.md. But it seems quite outdated so I’ll need some help to reach the end. As I am doing so, I update this file in a branch: Natron/INSTALL_WINDOWS.md at RB-2.3 · eliemichel/Natron · GitHub

Current updates are:

  1. First thing: the MINGW-packages seems to be now embedded inside the repo, so no need to clone them from somewhere else.
  2. Then since I ran into the trouble of searching for the actual name of each of the packages, I wrote down the full pacman commands.

Now, my current issue: the list of dependencies to install manually bothers me. All of those dependencies are available in the repos, but with some version mismatches. So I first tried to build with the deps got with pacman, but there is an error when building openfx-io which is related to ffmpeg.

Since the version in the repos is 4.0 and the version required in the instruction file is 3.0 I though this was the problem. I built ffmpeg 3.0.2 from source and installed it, uninstalled ffpmeg 4.0 and tried again to build openfx-io. New and different error.

So which version of ffmpeg should I use? Neither the version in mingw packages nor the one specified in the instruction file works. And the readme of openfx-io simply does not specify the required version.

Hi Élie,
checkout tools/jenkins/README.md - it contains much more up-to-date information on how to setup a build environment. Forget about almost everything in INSTALL_WINDOWS.md, and update it with relevant info from tools/jenkins/README.md - I’ll then gladly accept your pull request!
ffmpeg is 4.0 in the official builds. we are using a GPL2 ffmpeg, but you can also build a GPL3 ffmpeg (which does not bring a lot more)

and the MINGW-packages in the sources does not replace https://github.com/Alexpux/MINGW-packages but complements it. the script in jenkins/include/script/build-Windows-sdk.sh should build and install these extra packages.

Then, to build Natron and all of its plugins, you can start with the script from the tools/jenkins dir. The only thing that’s not yet in the build dir is breakpad (crash-reporter) integration, so remove CONFIG+=enable-breakpad from the qmake command line. I will add the Breakpad stuff soon, but it will be useless unless you intend to build official releases

Hi Frédéric, thanks for pointing me to the right place. I did not have anything missing when installing dependencies without including Alexpux packages… Should I be worried?

Then, I struggle a bit to understand what is really needed and what is for CI in those scripts. I cannot afford to dedicate my whole machine to Natron and so I’d rather do the configuration part manually than running setup scripts in admin mode and potentially break my other projects. So I did install the libraries by hand.

My current issue is still about ffmpeg while building openfx-io. More specifically, I get the folloging error:

 MINGW64 /e/SourceCode/Natron-openfx-io
$ make BITS=64 CONFIG=relwithdebinfo CXXFLAGS_ADD=-fopenmp LDFLAGS_ADD=-fopenmp
(cd IO && make )
make[1] : on entre dans le répertoire « /e/SourceCode/Natron-openfx-io/IO »
  CXX      MINGW64_NT-10.0-64-relwithdebinfo/WriteFFmpeg.o
In file included from ../FFmpeg/WriteFFmpeg.cpp:116:0:
../SupportExt/tinythread.h:540:0: warning: "ATOMIC_FLAG_INIT" redefined
 #define ATOMIC_FLAG_INIT 0

In file included from E:/msys64/mingw64/include/c++/7.3.0/bits/shared_ptr_atomic.h:33:0,
                 from E:/msys64/mingw64/include/c++/7.3.0/memory:82,
                 from ../openfx/Support/include/ofxsImageEffect.h:48,
                 from ../IOSupport/IOUtility.h:51,
                 from ../FFmpeg/WriteFFmpeg.cpp:78:
E:/msys64/mingw64/include/c++/7.3.0/bits/atomic_base.h:157:0: note: this is the location of the previous definition
 #define ATOMIC_FLAG_INIT { 0 }

../FFmpeg/WriteFFmpeg.cpp:2858:2: error: #error "Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead."
 #error "Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead."
  ^~~~~
In file included from ../FFmpeg/WriteFFmpeg.cpp:86:0:
../FFmpeg/FFmpegFile.h: In member function 'int FFmpegFile::Stream::getCodecDelay() const':
../FFmpeg/FFmpegFile.h:268:53: error: 'CODEC_CAP_DELAY' was not declared in this scope
             return ( ( (_videoCodec->capabilities & CODEC_CAP_DELAY) ? _codecContext->delay : 0 )
                                                     ^~~~~~~~~~~~~~~
../FFmpeg/FFmpegFile.h:268:53: note: suggested alternative: 'AV_CODEC_CAP_DELAY'
             return ( ( (_videoCodec->capabilities & CODEC_CAP_DELAY) ? _codecContext->delay : 0 )
                                                     ^~~~~~~~~~~~~~~
                                                     AV_CODEC_CAP_DELAY

The only ffmpeg version I have installed is the 4.0:

MINGW64 /e/SourceCode/Natron-openfx-io
$ pacman -Qs ffmpeg
local/mingw-w64-x86_64-ffmpeg 4.0-1
    Complete and free Internet live audio and video broadcasting solution (mingw-w64)

Also note that I’m trying to build the last stable version, namely the git tag 2.3. Any idea what this comes from? Is it that for some reason it found another version on my system? How may I check? Is the 2.3 also uses ffmpeg 4.0 or did it change since then? In this case, which version should I try to build in order to test my toolchain?

should be defined in FFmpegCompat.h.

#   if LIBAVCODEC_VERSION_INT >= AV_VERSION_INT(58, 0, 0)
/* ffmpeg 4.0 removed these */
#define CODEC_CAP_DELAY           AV_CODEC_CAP_DELAY
#define CODEC_CAP_DR1             AV_CODEC_CAP_DR1
#define CODEC_FLAG_GLOBAL_HEADER  AV_CODEC_FLAG_GLOBAL_HEADER
#   endif

and libavformat/version.h should define FF_API_LAVF_AVCTX, so you should not get the AVStream.codec error (see WriteFFmpeg.cpp source code).
You probably have a bleeding-edge ffmpeg installed somewhere. Chexck oput the output of these two commands, here are the versions that correspond to ffmpeg 4.0:

$ pkg-config --modversion libavformat
58.12.100
$ pkg-config --modversion libavcodec
58.18.100

I guess these are version 59 at your place.

I have the exact same version of libavformat and libavcodec than you do.

CODEC_CAP_DELAY is not defined in the 2.3.10 tagged commit: openfx-io/FFmpegCompat.h at Natron-2.3.10 · NatronGitHub/openfx-io · GitHub

So I’m guessing that the project moved from ffmpeg 3 to ffmpeg 4 after the 2.3.10. So I’m asking again: which version of Natron would you recommend me to build? How stable is master? I am looking for a stable enough version because I want to add some small stabilization fixes at least for a current project.

Yes, sure, use the master branch for the plugins and RB-2.3 for the main program.
ffmpeg 4.0 was released just after 2.3.10.
I will release 2.3.11 this week. I was only hoping to release it with flatpak Flatpak package · Issue #258 · NatronGitHub/Natron · GitHub

Thanks! So I successfully built openfx repositories, but now I have a problem with PySide. I installed it using pacman:

$ pacman -Qs pyside
local/mingw-w64-x86_64-pyside-common-qt4 1.2.2-3
    Provides LGPL Qt bindings for Python and related tools for binding generation (Common files)(mingw-w64)
local/mingw-w64-x86_64-pyside-tools-common-qt4 1.2.2-3
    PySide lupdate, rcc, and uic development tools (Common Files) (mingw-w64)
local/mingw-w64-x86_64-python2-pyside-qt4 1.2.2-3
    Provides LGPL Qt bindings for Python and related tools for binding generation (Python2)(mingw-w64)

But when running qmake -r in from my copy of Natron repo, I get:

$ qmake -r
Reading E:/SourceCode/Natron/HostSupport/HostSupport.pro
Reading E:/SourceCode/Natron/libs/gflags/gflags.pro
Reading E:/SourceCode/Natron/libs/glog/glog.pro
Reading E:/SourceCode/Natron/libs/ceres/ceres.pro
Reading E:/SourceCode/Natron/libs/libmv/libmv.pro
Reading E:/SourceCode/Natron/libs/openMVG/openMVG.pro
Reading E:/SourceCode/Natron/libs/qhttpserver/qhttpserver.pro
 Reading E:/SourceCode/Natron/libs/qhttpserver/src/src.pro
Reading E:/SourceCode/Natron/libs/hoedown/hoedown.pro
Reading E:/SourceCode/Natron/libs/libtess/libtess.pro
Reading E:/SourceCode/Natron/Engine/Engine.pro
Project MESSAGE: You did not select a config option for the build. Defaulting to Devel. You can choose among (snapshot | alpha | beta | RC | stable | custombuild). For custombuild you need to define the environment variable BUILD_USER_NAME. Also you can give a revision number to the version of Natron with the environment variable BUILD_NUMBER (e.g: RC1, RC2 etc...)
Project MESSAGE: Compiling in DEBUG mode.
Project ERROR: Package pyside-py2 not found

NB: I also tried to define the BUILD_USER_NAME variable but it did not remove the warning.

But pyside is found in a Python repl though:

$ python
Python 2.7.14 (default, Jan 24 2018, 14:36:54)  [GCC 7.2.0 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import PySide
>>>

I tried to install pyside using pip, but it failed to find qt (MSYS terminal) or to find linux-like python lib (MINGW terminal) and I think it is recommended to install it without pip because it relies on Qt.

Do you think that the problem comes from my MSYS setup? Or a missing library? Thanks again for your support.

try:
pacman -Ql mingw-w64-x86_64-python2-pyside-qt4

it should tell you that /mingw64/lib/pkgconfig/pyside-py2.pc is part of the package.

now try
pkg-config --libs pyside-py2
it should give -LC:/msys64/mingw64/lib -lpyside-python2.7 -lpython2.7.dll -lshiboken-python2.7

if these are OK, then qmake should find pyside-py2

/mingw64/lib/pkgconfig/pyside-py2.pc was indeed installed, but the problem came from Shiboken.

$ pkg-config --libs pyside-py2
Package shiboken was not found in the pkg-config search path.
Perhaps you should add the directory containing `shiboken.pc'
to the PKG_CONFIG_PATH environment variable
Package 'shiboken', required by 'pyside-py2', not found

So I did the same for shiboken: pacman -Ql mingw-w64-x86_64-python2-shiboken-qt4. Turns out that the package contains a shiboken-py2.pc but no shiboken.pc and PySide expects the latter. I read tools/MINGW-packages/mingw-w64-pyside-qt4 and I can see in it that any reference to shiboken in PySide package should be rename into shiboken-py2. So I apparently have an inconsistency between my shiboken and pyside packages, one coming from custom repos and not the other one or something like this. I ended up dirty patching my system:

ln -s /mingw64/lib/pkgconfig/shiboken-py2.pc /mingw64/lib/pkgconfig/shiboken.pc

But I’m not happy with it. It’s not something I can release officially in a building How To.

Anyway, I kept on going, and ran qmake. It was not clear which config I should use, since most of them were plotting a warning saying they are designed to be ran at Inria. In particular, I did not manage to set an installation target. Which qmake options would you recommend for an optimized release-like build?

For now I build in the defautl devel mode, and specify an installation path:

qmake -r target.path=/e/SourceCode/Natron/bin

This now runs well, so I go for make then. But at linking time I ran into this issue: Alexpux’ MINGW-packages #1670, with pyside package. But greping my system for some mention to C:\building did not return anything else than the makefiles generated by qmake. So I manually edited those in order to keep going, but again it’s not something I could advise anybody to do.

After that, the make ended correctly. So I ran make install, and checked out my install dir. There were only three elements in it:

Natron.exe
natron-python.exe
NatronRenderer.exe

I copied the libgcc dll it was asking for, but then could not start any of those exe:

pasdemarre

I would rather edit pyside-py2.pc to make it refer to shiboken-py2 rather than shiboken as a dependency.

As for the other error, ldd is your friend to find out DLL issues. You probably put the wrong libgcc.dll

You can also use Dependency Walker

Hello,

I’ve been trying further again to build Natron, and fixed this dll issue, but I’m now facing a new issue: installing PySide. It seems that it is not possible to install PySide on Msys. How is it so? How do you youse to work this around to build Natron for Windows?

É.

1 Like

Hello, I was also studying natron recently, but I encountered trouble during the compilation process, how did you install the shiboken and pyside packages?

We maintain a custom MSYS2 snapshot + repo for Natron, see Natron/tools/MINGW-packages at RB-2.3 · NatronGitHub/Natron · GitHub for more information.

Hi.

I tried if I could build Natron 2.3 myself on Windows 10, but failed. I have no experience on this and not sure if the end result is installation package, directly running application or something else, but tried to follow instructions to see. I also might very well miss some necessary software if instructions expect prior development experience and knowlege.

But in case someone has time and patience, I’d appreciete help to guide me through this.

I have own fork in Github and synced to my computer. Below are steps that I did and where it failed:

  • Installed MSYS2 per instructions (Natron/README.md at RB-2.3 · NatronGitHub/Natron · GitHub)
  • Checked if OpenFX API is up to date (git submodule update -i --recursive)
  • Created config.pri file with paths to LIBS -folder
  • Did NOT build nodes (thought maybe not necessary for test)
  • BUILD FAILED (see below)

Saku@DESKTOP-23R7QK7 MINGW64 /d/PROJECTS/Natron_dev/Development/Natron
$ qmake -r
Reading D:/PROJECTS/Natron_dev/Development/Natron/HostSupport/HostSupport.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/gflags/gflags.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/glog/glog.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/ceres/ceres.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/libmv/libmv.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/openMVG/openMVG.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/qhttpserver/qhttpserver.p ro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/qhttpserver/src/src.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/hoedown/hoedown.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/libs/libtess/libtess.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/Engine/Engine.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/Renderer/Renderer.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/Gui/Gui.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/Tests/Tests.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/PythonBin/PythonBin.pro
Reading D:/PROJECTS/Natron_dev/Development/Natron/App/App.pro
Project MESSAGE: You did not select a config option for the build. Defaulting to Devel. You can choose among (snapshot | alpha | beta | RC | stable | custombuil d). For custombuild you need to define the environment variable BUILD_USER_NAME. Also you can give a revision number to the version of Natron with the environme nt variable BUILD_NUMBER (e.g: RC1, RC2 etc…)
Project MESSAGE: Compiling in RELEASE mode.
WARNING: Failure to find: release/Natron_res.o
WARNING: Failure to find: debug/Natron_res.o
WARNING: target.path is not defined: install target not created

WARNING: target.path is not defined: install target not created

WARNING: target.path is not defined: install target not created

WARNING: target.path is not defined: install target not created

WARNING: Failure to find: release/Natron_res.o
WARNING: Failure to find: debug/Natron_res.o
WARNING: target.path is not defined: install target not created

WARNING: target.path is not defined: install target not created

You didn’t start the build? type make -jX (X=amount of threads you have)

1 Like

You are right. I didn’t actually build, but stopped after creating Makefile when seeing warnings, so the last step I mentioned was incorrect.

Now I built it and looked like it finished without errors. Had some warnings though during the process. Now I have no idea where the software was built to? :slight_smile:

In Makefile I don’t see any target specified and in my earlier post it had WARNINGS that install target not created. Not sure if there should be a path or not, but here is the beginning of file to see.

first: make_default
MAKEFILE      = Makefile
QMAKE         = c:/msys64-20180531/mingw64/bin/qmake.exe
DEL_FILE      = rm -f
CHK_DIR_EXISTS= test -d
MKDIR         = mkdir -p
COPY          = cp -f
COPY_FILE     = $(COPY)
COPY_DIR      = cp -rf
INSTALL_FILE  = $(COPY_FILE)
INSTALL_PROGRAM = $(COPY_FILE)
INSTALL_DIR   = $(COPY_DIR)
DEL_FILE      = rm -f
SYMLINK       = 
DEL_DIR       = rmdir
MOVE          = mv
CHK_DIR_EXISTS= test -d
MKDIR         = mkdir -p
SUBTARGETS    =  \
		sub-HostSupport \
		sub-libs-gflags \
...
...

The warnings are normal, I test the SDK regularly, it works :slight_smile:

The Natron binary should be in the App folder where you built it.

1 Like

I see. Unfortunately the /Natron/App/release/ -folder is empty. But maybe I’m not far away to get it working.

/Natron/App/ -folder has Makefile.Debug and Makefile.Release files from yesterday created at the time I made the Makefile and has debug and release folders, but those both folders are empty.

In some other /Natron/“folder” -locations have release folders generated during the build with .o and .cpp files, but I don’t know if that is correct during the building.

/Natron/Gui/release/
/Natron/Renderer/release/
/Natron/Engine/release/

I copy here also snippet of the very last part of my build info if that looks like to be correct or if right away can see some problems.

In file included from ../../Natron/Engine/PyParameter.h:41:0,
                 from ../Engine/PyTracker.h:38,
                 from ../Engine/NatronEngine/natronengine_python.h:27,
                 from NatronGui/natrongui_python.h:29,
                 from NatronGui/natrongui_module_wrapper.cpp:13:
../../Natron/Engine/KnobTypes.h:39:1: warning: unknown option after '#pragma GCC diagnostic' kind [-Wpragmas]
 GCC_ONLY_DIAG_OFF(class-memaccess)
 ^~~~~~
g++ -c -include pch/pch.h -pipe -include Python.h -ftemplate-depth-1024 -Wno-deprecated-declarations -fpermissive -Wno-int-in-bool-context -Wno-expansion-to-defined -municode -O2 -g -mms-bitfields -Wall -Wextra -Wall -Wextra -Wno-multichar -Winit-self -Wno-long-long -fexceptions -mthreads -frtti -DUNICODE -DOFX_EXTENSIONS_NUKE -DOFX_EXTENSIONS_TUTTLE -DOFX_EXTENSIONS_VEGAS -DOFX_SUPPORTS_PARAMETRIC -DOFX_EXTENSIONS_TUTTLE -DOFX_EXTENSIONS_NATRON -DOFX_EXTENSIONS_RESOLVE -DOFX_SUPPORTS_OPENGLRENDER -DOFX_SUPPORTS_MULTITHREAD -DOFX_SUPPORTS_DIALOG -DQT_NO_CAST_FROM_ASCII -DNDEBUG -DNATRON_BUILD_NUMBER=0 -DWINDOWS -DCOMPILED_FROM_DSP -DXML_STATIC -DNOMINMAX -D_UNICODE -DUNICODE -DQHTTP_SERVER_STATIC -DQT_DLL -DQT_NO_DEBUG -DQT_NO_KEYWORDS -DQT_OPENGL_LIB -DQT_GUI_LIB -DQT_NETWORK_LIB -DQT_CORE_LIB -DQT_THREAD_SUPPORT -DQT_SHARED -I'c:/msys64-20180531/mingw64/include/qt4/QtCore' -I'c:/msys64-20180531/mingw64/include/qt4/QtNetwork' -I'c:/msys64-20180531/mingw64/include/qt4/QtGui' -I'c:/msys64-20180531/mingw64/include/qt4/QtOpenGL' -I'c:/msys64-20180531/mingw64/include/qt4' -I'../Global/gladRel/include' -I'c:/msys64-20180531/mingw64/include/PySide/QtCore' -I'c:/msys64-20180531/mingw64/include/PySide/QtGui' -I'../libs/OpenFX/include' -I'../libs/OpenFX_extensions' -I'../libs/OpenFX/HostSupport/include' -I'../../Natron' -I'../libs/SequenceParsing' -I'../Engine' -I'../Engine/NatronEngine' -I'../Global' -I'../libs/qhttpserver/src' -I'.' -I'NatronGui' -I'c:/msys64-20180531/mingw64/include/cairo' -I'c:/msys64-20180531/mingw64/include/pixman-1' -I'c:/msys64-20180531/mingw64/include' -I'c:/msys64-20180531/mingw64/include/freetype2' -I'c:/msys64-20180531/mingw64/include/harfbuzz' -I'c:/msys64-20180531/mingw64/include/glib-2.0' -I'c:/msys64-20180531/mingw64/lib/glib-2.0/include' -I'c:/msys64-20180531/mingw64/include/libpng16' -I'c:/msys64-20180531/mingw64/include/python2.7' -I'c:/msys64-20180531/mingw64/include/shiboken' -I'c:/msys64-20180531/mingw64/include/PySide' -I'c:/msys64-20180531/mingw64/include/qt4/ActiveQt' -I'release' -I'c:/msys64-20180531/mingw64/share/qt4/mkspecs/default' -o release/qrc_GuiResources.o release/qrc_GuiResources.cpp
ViewerTab.cpp: In constructor 'Natron::ViewerTab::ViewerTab(const std::__cxx11::list<boost::shared_ptr<Natron::NodeGui> >&, const std::__cxx11::list<boost::shared_ptr<Natron::NodeGui> >&, Natron::Gui*, Natron::ViewerInstance*, QWidget*)':
ViewerTab.cpp:100:1: note: variable tracking size limit exceeded with -fvar-tracking-assignments, retrying without
 ViewerTab::ViewerTab(const std::list<NodeGuiPtr> & existingNodesContext,
 ^~~~~~~~~
ar -M < object_script.libGui.Release
make[2]: Leaving directory '/d/PROJECTS/Natron_dev/Development/Natron/Gui'
make[1]: Leaving directory '/d/PROJECTS/Natron_dev/Development/Natron/Gui'

Saku@DESKTOP-23R7QK7 MINGW64 /d/PROJECTS/Natron_dev/Development/Natron
$
1 Like