Building Natron on Windows from scratch


(Élie Michel) #1

Hi! I am trying to build Natron from scratch and run into a couple of interrogations.

First of all, I had to figure out which repository I should use, because natron.fr points to MrKepzie’s repo while the official one actually is NatronGitHub/Natron.

I am building on Windows (with MSYS2), and so logically had a look at INSTALL_WINDOWS.md. But it seems quite outdated so I’ll need some help to reach the end. As I am doing so, I update this file in a branch: https://github.com/eliemichel/Natron/blob/RB-2.3/INSTALL_WINDOWS.md

Current updates are:

  1. First thing: the MINGW-packages seems to be now embedded inside the repo, so no need to clone them from somewhere else.
  2. Then since I ran into the trouble of searching for the actual name of each of the packages, I wrote down the full pacman commands.

Now, my current issue: the list of dependencies to install manually bothers me. All of those dependencies are available in the repos, but with some version mismatches. So I first tried to build with the deps got with pacman, but there is an error when building openfx-io which is related to ffmpeg.

Since the version in the repos is 4.0 and the version required in the instruction file is 3.0 I though this was the problem. I built ffmpeg 3.0.2 from source and installed it, uninstalled ffpmeg 4.0 and tried again to build openfx-io. New and different error.

So which version of ffmpeg should I use? Neither the version in mingw packages nor the one specified in the instruction file works. And the readme of openfx-io simply does not specify the required version.


OpenFX Plugin Development/Debugging on Windows
(Frédéric Devernay) #2

Hi Élie,
checkout tools/jenkins/README.md - it contains much more up-to-date information on how to setup a build environment. Forget about almost everything in INSTALL_WINDOWS.md, and update it with relevant info from tools/jenkins/README.md - I’ll then gladly accept your pull request!
ffmpeg is 4.0 in the official builds. we are using a GPL2 ffmpeg, but you can also build a GPL3 ffmpeg (which does not bring a lot more)


(Frédéric Devernay) #3

and the MINGW-packages in the sources does not replace https://github.com/Alexpux/MINGW-packages but complements it. the script in jenkins/include/script/build-Windows-sdk.sh should build and install these extra packages.


(Frédéric Devernay) #4

Then, to build Natron and all of its plugins, you can start with the script from the tools/jenkins dir. The only thing that’s not yet in the build dir is breakpad (crash-reporter) integration, so remove CONFIG+=enable-breakpad from the qmake command line. I will add the Breakpad stuff soon, but it will be useless unless you intend to build official releases


(Élie Michel) #5

Hi Frédéric, thanks for pointing me to the right place. I did not have anything missing when installing dependencies without including Alexpux packages… Should I be worried?

Then, I struggle a bit to understand what is really needed and what is for CI in those scripts. I cannot afford to dedicate my whole machine to Natron and so I’d rather do the configuration part manually than running setup scripts in admin mode and potentially break my other projects. So I did install the libraries by hand.

My current issue is still about ffmpeg while building openfx-io. More specifically, I get the folloging error:

 MINGW64 /e/SourceCode/Natron-openfx-io
$ make BITS=64 CONFIG=relwithdebinfo CXXFLAGS_ADD=-fopenmp LDFLAGS_ADD=-fopenmp
(cd IO && make )
make[1] : on entre dans le répertoire « /e/SourceCode/Natron-openfx-io/IO »
  CXX      MINGW64_NT-10.0-64-relwithdebinfo/WriteFFmpeg.o
In file included from ../FFmpeg/WriteFFmpeg.cpp:116:0:
../SupportExt/tinythread.h:540:0: warning: "ATOMIC_FLAG_INIT" redefined
 #define ATOMIC_FLAG_INIT 0

In file included from E:/msys64/mingw64/include/c++/7.3.0/bits/shared_ptr_atomic.h:33:0,
                 from E:/msys64/mingw64/include/c++/7.3.0/memory:82,
                 from ../openfx/Support/include/ofxsImageEffect.h:48,
                 from ../IOSupport/IOUtility.h:51,
                 from ../FFmpeg/WriteFFmpeg.cpp:78:
E:/msys64/mingw64/include/c++/7.3.0/bits/atomic_base.h:157:0: note: this is the location of the previous definition
 #define ATOMIC_FLAG_INIT { 0 }

../FFmpeg/WriteFFmpeg.cpp:2858:2: error: #error "Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead."
 #error "Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead."
  ^~~~~
In file included from ../FFmpeg/WriteFFmpeg.cpp:86:0:
../FFmpeg/FFmpegFile.h: In member function 'int FFmpegFile::Stream::getCodecDelay() const':
../FFmpeg/FFmpegFile.h:268:53: error: 'CODEC_CAP_DELAY' was not declared in this scope
             return ( ( (_videoCodec->capabilities & CODEC_CAP_DELAY) ? _codecContext->delay : 0 )
                                                     ^~~~~~~~~~~~~~~
../FFmpeg/FFmpegFile.h:268:53: note: suggested alternative: 'AV_CODEC_CAP_DELAY'
             return ( ( (_videoCodec->capabilities & CODEC_CAP_DELAY) ? _codecContext->delay : 0 )
                                                     ^~~~~~~~~~~~~~~
                                                     AV_CODEC_CAP_DELAY

The only ffmpeg version I have installed is the 4.0:

MINGW64 /e/SourceCode/Natron-openfx-io
$ pacman -Qs ffmpeg
local/mingw-w64-x86_64-ffmpeg 4.0-1
    Complete and free Internet live audio and video broadcasting solution (mingw-w64)

Also note that I’m trying to build the last stable version, namely the git tag 2.3. Any idea what this comes from? Is it that for some reason it found another version on my system? How may I check? Is the 2.3 also uses ffmpeg 4.0 or did it change since then? In this case, which version should I try to build in order to test my toolchain?


(Frédéric Devernay) #6

should be defined in FFmpegCompat.h.

#   if LIBAVCODEC_VERSION_INT >= AV_VERSION_INT(58, 0, 0)
/* ffmpeg 4.0 removed these */
#define CODEC_CAP_DELAY           AV_CODEC_CAP_DELAY
#define CODEC_CAP_DR1             AV_CODEC_CAP_DR1
#define CODEC_FLAG_GLOBAL_HEADER  AV_CODEC_FLAG_GLOBAL_HEADER
#   endif

and libavformat/version.h should define FF_API_LAVF_AVCTX, so you should not get the AVStream.codec error (see WriteFFmpeg.cpp source code).
You probably have a bleeding-edge ffmpeg installed somewhere. Chexck oput the output of these two commands, here are the versions that correspond to ffmpeg 4.0:

$ pkg-config --modversion libavformat
58.12.100
$ pkg-config --modversion libavcodec
58.18.100

I guess these are version 59 at your place.


(Élie Michel) #7

I have the exact same version of libavformat and libavcodec than you do.

CODEC_CAP_DELAY is not defined in the 2.3.10 tagged commit: https://github.com/NatronGitHub/openfx-io/blob/Natron-2.3.10/FFmpeg/FFmpegCompat.h

So I’m guessing that the project moved from ffmpeg 3 to ffmpeg 4 after the 2.3.10. So I’m asking again: which version of Natron would you recommend me to build? How stable is master? I am looking for a stable enough version because I want to add some small stabilization fixes at least for a current project.


(Frédéric Devernay) #8

Yes, sure, use the master branch for the plugins and RB-2.3 for the main program.
ffmpeg 4.0 was released just after 2.3.10.
I will release 2.3.11 this week. I was only hoping to release it with flatpak https://github.com/NatronGitHub/Natron/issues/258


(Élie Michel) #9

Thanks! So I successfully built openfx repositories, but now I have a problem with PySide. I installed it using pacman:

$ pacman -Qs pyside
local/mingw-w64-x86_64-pyside-common-qt4 1.2.2-3
    Provides LGPL Qt bindings for Python and related tools for binding generation (Common files)(mingw-w64)
local/mingw-w64-x86_64-pyside-tools-common-qt4 1.2.2-3
    PySide lupdate, rcc, and uic development tools (Common Files) (mingw-w64)
local/mingw-w64-x86_64-python2-pyside-qt4 1.2.2-3
    Provides LGPL Qt bindings for Python and related tools for binding generation (Python2)(mingw-w64)

But when running qmake -r in from my copy of Natron repo, I get:

$ qmake -r
Reading E:/SourceCode/Natron/HostSupport/HostSupport.pro
Reading E:/SourceCode/Natron/libs/gflags/gflags.pro
Reading E:/SourceCode/Natron/libs/glog/glog.pro
Reading E:/SourceCode/Natron/libs/ceres/ceres.pro
Reading E:/SourceCode/Natron/libs/libmv/libmv.pro
Reading E:/SourceCode/Natron/libs/openMVG/openMVG.pro
Reading E:/SourceCode/Natron/libs/qhttpserver/qhttpserver.pro
 Reading E:/SourceCode/Natron/libs/qhttpserver/src/src.pro
Reading E:/SourceCode/Natron/libs/hoedown/hoedown.pro
Reading E:/SourceCode/Natron/libs/libtess/libtess.pro
Reading E:/SourceCode/Natron/Engine/Engine.pro
Project MESSAGE: You did not select a config option for the build. Defaulting to Devel. You can choose among (snapshot | alpha | beta | RC | stable | custombuild). For custombuild you need to define the environment variable BUILD_USER_NAME. Also you can give a revision number to the version of Natron with the environment variable BUILD_NUMBER (e.g: RC1, RC2 etc...)
Project MESSAGE: Compiling in DEBUG mode.
Project ERROR: Package pyside-py2 not found

NB: I also tried to define the BUILD_USER_NAME variable but it did not remove the warning.

But pyside is found in a Python repl though:

$ python
Python 2.7.14 (default, Jan 24 2018, 14:36:54)  [GCC 7.2.0 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import PySide
>>>

I tried to install pyside using pip, but it failed to find qt (MSYS terminal) or to find linux-like python lib (MINGW terminal) and I think it is recommended to install it without pip because it relies on Qt.

Do you think that the problem comes from my MSYS setup? Or a missing library? Thanks again for your support.


(Frédéric Devernay) #10

try:
pacman -Ql mingw-w64-x86_64-python2-pyside-qt4

it should tell you that /mingw64/lib/pkgconfig/pyside-py2.pc is part of the package.

now try
pkg-config --libs pyside-py2
it should give -LC:/msys64/mingw64/lib -lpyside-python2.7 -lpython2.7.dll -lshiboken-python2.7

if these are OK, then qmake should find pyside-py2


(Élie Michel) #11

/mingw64/lib/pkgconfig/pyside-py2.pc was indeed installed, but the problem came from Shiboken.

$ pkg-config --libs pyside-py2
Package shiboken was not found in the pkg-config search path.
Perhaps you should add the directory containing `shiboken.pc'
to the PKG_CONFIG_PATH environment variable
Package 'shiboken', required by 'pyside-py2', not found

So I did the same for shiboken: pacman -Ql mingw-w64-x86_64-python2-shiboken-qt4. Turns out that the package contains a shiboken-py2.pc but no shiboken.pc and PySide expects the latter. I read tools/MINGW-packages/mingw-w64-pyside-qt4 and I can see in it that any reference to shiboken in PySide package should be rename into shiboken-py2. So I apparently have an inconsistency between my shiboken and pyside packages, one coming from custom repos and not the other one or something like this. I ended up dirty patching my system:

ln -s /mingw64/lib/pkgconfig/shiboken-py2.pc /mingw64/lib/pkgconfig/shiboken.pc

But I’m not happy with it. It’s not something I can release officially in a building How To.

Anyway, I kept on going, and ran qmake. It was not clear which config I should use, since most of them were plotting a warning saying they are designed to be ran at Inria. In particular, I did not manage to set an installation target. Which qmake options would you recommend for an optimized release-like build?

For now I build in the defautl devel mode, and specify an installation path:

qmake -r target.path=/e/SourceCode/Natron/bin

This now runs well, so I go for make then. But at linking time I ran into this issue: Alexpux’ MINGW-packages #1670, with pyside package. But greping my system for some mention to C:\building did not return anything else than the makefiles generated by qmake. So I manually edited those in order to keep going, but again it’s not something I could advise anybody to do.

After that, the make ended correctly. So I ran make install, and checked out my install dir. There were only three elements in it:

Natron.exe
natron-python.exe
NatronRenderer.exe

I copied the libgcc dll it was asking for, but then could not start any of those exe:

pasdemarre


(Frédéric Devernay) #12

I would rather edit pyside-py2.pc to make it refer to shiboken-py2 rather than shiboken as a dependency.

As for the other error, ldd is your friend to find out DLL issues. You probably put the wrong libgcc.dll

You can also use Dependency Walker