Compiling Natron yourself using the docker SDK

After installing docker, it’s as easy as:

mkdir builds
docker run -it --rm --mount src="$(pwd)/builds",target=/home/builds_archive,type=bind natrongithub/natron-sdk:latest

and the results are in a directory called builds

The docker image is available from the docker hub https://hub.docker.com/r/natrongithub/natron-sdk

The builds can be easily customized, for example to use you own git repo for Natron:

mkdir builds
docker run -it --rm --mount src="$(pwd)/builds",target=/home/builds_archive,type=bind --env GIT_URL=https://github.com/MyGitLogin/Natron.git --env GIT_BRANCH=mybranch natrongithub/natron-sdk:latest

If you want to make a custom docker image based on anything else than centos6, just modify the base image in the Dockerfile generated by this build script:

2 Likes

Thanks! I was able to make it run and produce a valid tgz installation yesterday.

I am just trying to re-run it and it got stuck forever in a test :

Loading project: /home/tmp/Natron-Tests64/GNUVolador/gnu.ntp
(‘encoder=’, u’fr.inria.openfx.WriteOIIO’)
(‘ocioInputSpace=’, u’linear/Linear’)
(‘ocioOutputSpace=’, u’sRGB’)
script_write_node ==> Rendering started
script_write_node ==> Frame: 1, Progress: 11.1%, 0.7 Fps, Time Remaining: 11 seconds

I will retry running the docker build again but I would like to know if some test are known for being unstable?

Regards,

Yes, this test gets stuck when I run it too (but the test script kills it after 30mn).
This may be due to the limited number of chores available in docker.
The Baymax test also fails but I did not have time to look into that.
Once you have Natron installed on a real machine you can also rerun the tests from the repo Natron-Tests. Does the gnuvolador pass? Does Baymax pass?

I started 4 build of different commit version since 3 days ago to see if this was a recent problem.

1 out of those 4 build didn’t get stuck and I got :
2019-08-01 20:30:50 *** END unit tests -> 0
Unit tests: OK

The 3 other build are stuck at Baymax : two in Natron2_BayMaxProject_20_fixed_switched.ntp and one in /home/tmp/Natron-Tests64/BayMax/Natron2_BayMaxProject_20_fixed.ntp

Since one of those stuck build is the same git commit I was able to build 2 day ago I guess there is something unstable. I will try to run the test individually if I get some time to see if I can reproduce the problem outside of docker

Running the Natron2_BayMaxProject_20_fixed.ntp in GUI with my docker build or the 2.3.15 rc6 I got 2 issues:

There is a lots of warning when loading the project :

[17:12:19.864] Project: WARNING: The node LensDistortion (net.sf.openfx.LensDistortion) version 2.0 was found in the script but was loaded with version 4.0 instead.
[17:12:20.416] Premult1: Could not find a parameter named Source_channels
[17:12:20.417] Shufflecopy1: Could not find a parameter named outputChannelsChoice

The video is not right (I saw only the top of the attic and it is warped).

I am able to get the playback to get stuck. It play some frame and at some point it just stop and I can’t stop start the playback again. It is like if the play button didn’t work.

If I use the official Natron 2.3.14 I didn’t get any of those issue.

I make sure to delete all Natron caches before trying different version
rm ~/.cache/INRIA/Natron/* -rf

NOTE : I was sometime able to get the playback to stop responding on my own test project when using other build than the 2.3.14 but was not able to find a good way to reproduce it reliably. Seems this project is big enough to cause the problem to show more

I tried building older version of Natron using the docker but was not able to go very far back in time because of build issue.

Is there a place where some portable intermediary build between version 2.3.14 and version 2.3.15rc6 could be found? I could try to see between which version the problem with BayMax appear.

The project file from Natron_2.3.12_BayMax.zip works fine on latest snapshot available on GitHub. Must be something wrong with the ‘ntp’ in the Tests?

Natron (and or plugins) should be fixed to load Natron2_BayMaxProject_20_fixed.ntp

This reveals several issues which have to be fixed without changing the ntp file

Yves,

These warnings also appear in 2.3.14:

Summary

[10:35:35.450] Premult2: Could not find a parameter named Source_channels

[10:35:35.451] Premult1: Could not find a parameter named Source_channels

[10:35:35.451] Shuffle1: Could not find a parameter named outputChannelsChoice

[10:35:35.451] Shuffle1: Could not find a parameter named outputRChoice

[10:35:35.451] Shuffle1: Could not find a parameter named outputGChoice

[10:35:35.451] Shuffle1: Could not find a parameter named outputBChoice

[10:35:35.451] Shuffle1: Could not find a parameter named outputAChoice

[10:35:35.453] Shufflecopy1: Could not find a parameter named outputChannelsChoice

[10:35:35.453] Shufflecopy1: Could not find a parameter named outputRChoice

[10:35:35.453] Shufflecopy1: Could not find a parameter named outputGChoice

[10:35:35.453] Shufflecopy1: Could not find a parameter named outputBChoice

[10:35:35.453] Shufflecopy1: Could not find a parameter named outputAChoice

[10:35:35.892] Reset_Alpha_to_1: Could not find a parameter named outputChannelsChoice

[10:35:35.892] Reset_Alpha_to_1: Could not find a parameter named outputRChoice

[10:35:35.892] Reset_Alpha_to_1: Could not find a parameter named outputGChoice

[10:35:35.892] Reset_Alpha_to_1: Could not find a parameter named outputBChoice

[10:35:35.892] Reset_Alpha_to_1: Could not find a parameter named outputAChoice

[10:35:35.892] WriteOIIO1: Could not find a parameter named clipToProject

[10:35:35.892] WriteOIIO1: Could not find a parameter named outputChannelsChoice

[10:35:35.892] Shuffle1: Could not find a parameter named outputChannelsChoice

[10:35:35.892] Shuffle1: Could not find a parameter named outputRChoice

[10:35:35.892] Shuffle1: Could not find a parameter named outputGChoice

[10:35:35.892] Shuffle1: Could not find a parameter named outputBChoice

[10:35:35.892] Shuffle1: Could not find a parameter named outputAChoice

[10:35:35.893] LensDistortion: Could not find a parameter named k3

[10:35:35.893] LensDistortion: Could not find a parameter named p1

[10:35:35.893] LensDistortion: Could not find a parameter named p2

[10:35:35.893] Shuffle_CopyAlpha: Could not find a parameter named outputChannelsChoice

[10:35:35.893] Shuffle_CopyAlpha: Could not find a parameter named outputRChoice

[10:35:35.893] Shuffle_CopyAlpha: Could not find a parameter named outputGChoice

[10:35:35.893] Shuffle_CopyAlpha: Could not find a parameter named outputBChoice

[10:35:35.893] Shuffle_CopyAlpha: Could not find a parameter named outputAChoice

Here’s how to debug the issue:

  • load the ntp using 2.3.14 and 2.3.15 side-by-side, preferably not on the same system (use tow PCSs or VMs)
  • inspect each node output using the viewer, until you find where the two diverge.

Frederic,
I don’t see the warning in 2.3.14 on a centos 7 pc.
Thanks for the tips. I will post an update once I can confirm some of the things. So far I think there is 2 separate issues. If so, I will open bug report for each.

You can see the warnings in ing the error log (Display->Show Project Error Log)
You can also use the environment variable NATRON_DISK_CACHE_PATH to get two versions of Natron running simultaneously but using different disk caches (it was not documented, I’ll add a section in the doc on environment variables)

Ok thanks. I was able to compare the warning and found there is a new one. I open a bug for this issue : https://github.com/NatronGitHub/Natron/issues/416
I open issue https://github.com/NatronGitHub/Natron/issues/417 for the freeze rendering issue.

I will try to build intermediate version of Natron (from 2.3.14 to 2.3.15rc6) with a normal build and see if I can find the problematic commit. (assuming I can build easily older version of Natron)

The major change is when I fixed the the Python Gil handling

While trying to fix an issue I am trying to build a complete tgz from my own repo of Natron and branch and got some error :

docker run -it --rm --mount src="$(pwd)/builds",target=/home/builds_archive,type=bind --env GIT_URL=https://github.com/Ivanohe73/Natron.git --env GIT_BRANCH=rotopaint natrongithub/natron-sdk:latest

./launchBuildMain.sh: line 306: GIT_NATRON: unbound variable

I tried adding a
--env GIT_NATRON=https://github.com/Ivanohe73/Natron.git

but got error:
Unknown plug-in repository URL https://github.com/Ivanohe73/Natron.git

I also forked all the plug-in in case and tried adding other options like GIT_URL_OPENFX_IO , etc but I still get the Unknown plug-in repository error…

Thanks @rodlie !

So here is the updated way with docker

to use you own git repo for Natron:

mkdir builds
docker run -it --rm --mount src="$(pwd)/builds",target=/home/builds_archive,type=bind --env GIT_NATRON=https://github.com/MyGitLogin/Natron.git --env GIT_URL=https://github.com/MyGitLogin/Natron.git --env GIT_BRANCH=mybranch natrongithub/natron-sdk:latest /bin/bash

sed -c -i "s/^GIT_URL_NATRON_GITHUB=.*/GIT_URL_NATRON_GITHUB=\$GIT_NATRON/" gitRepositories.sh
./launchBuildMain.sh

Fixed in https://github.com/NatronGitHub/Natron/commit/a8cc6bebee798b5256d2a91bfb2b37236f6b9d40

If GIT_URL is an unknown Natron repository, GIT_URL_IS_NATRON=1 can be used to force a Natron build.