Compiling Natron yourself using the docker SDK

devernay · August 3, 2019, 5:28pm

Natron (and or plugins) should be fixed to load Natron2_BayMaxProject_20_fixed.ntp

This reveals several issues which have to be fixed without changing the ntp file

devernay · August 3, 2019, 5:39pm

Yves,

These warnings also appear in 2.3.14:

Summary

[10:35:35.450] Premult2: Could not find a parameter named Source_channels

[10:35:35.451] Premult1: Could not find a parameter named Source_channels

[10:35:35.451] Shuffle1: Could not find a parameter named outputChannelsChoice

[10:35:35.451] Shuffle1: Could not find a parameter named outputRChoice

[10:35:35.451] Shuffle1: Could not find a parameter named outputGChoice

[10:35:35.451] Shuffle1: Could not find a parameter named outputBChoice

[10:35:35.451] Shuffle1: Could not find a parameter named outputAChoice

[10:35:35.453] Shufflecopy1: Could not find a parameter named outputChannelsChoice

[10:35:35.453] Shufflecopy1: Could not find a parameter named outputRChoice

[10:35:35.453] Shufflecopy1: Could not find a parameter named outputGChoice

[10:35:35.453] Shufflecopy1: Could not find a parameter named outputBChoice

[10:35:35.453] Shufflecopy1: Could not find a parameter named outputAChoice

[10:35:35.892] Reset_Alpha_to_1: Could not find a parameter named outputChannelsChoice

[10:35:35.892] Reset_Alpha_to_1: Could not find a parameter named outputRChoice

[10:35:35.892] Reset_Alpha_to_1: Could not find a parameter named outputGChoice

[10:35:35.892] Reset_Alpha_to_1: Could not find a parameter named outputBChoice

[10:35:35.892] Reset_Alpha_to_1: Could not find a parameter named outputAChoice

[10:35:35.892] WriteOIIO1: Could not find a parameter named clipToProject

[10:35:35.892] WriteOIIO1: Could not find a parameter named outputChannelsChoice

[10:35:35.892] Shuffle1: Could not find a parameter named outputChannelsChoice

[10:35:35.892] Shuffle1: Could not find a parameter named outputRChoice

[10:35:35.892] Shuffle1: Could not find a parameter named outputGChoice

[10:35:35.892] Shuffle1: Could not find a parameter named outputBChoice

[10:35:35.892] Shuffle1: Could not find a parameter named outputAChoice

[10:35:35.893] LensDistortion: Could not find a parameter named k3

[10:35:35.893] LensDistortion: Could not find a parameter named p1

[10:35:35.893] LensDistortion: Could not find a parameter named p2

[10:35:35.893] Shuffle_CopyAlpha: Could not find a parameter named outputChannelsChoice

[10:35:35.893] Shuffle_CopyAlpha: Could not find a parameter named outputRChoice

[10:35:35.893] Shuffle_CopyAlpha: Could not find a parameter named outputGChoice

[10:35:35.893] Shuffle_CopyAlpha: Could not find a parameter named outputBChoice

[10:35:35.893] Shuffle_CopyAlpha: Could not find a parameter named outputAChoice

Here’s how to debug the issue:

load the ntp using 2.3.14 and 2.3.15 side-by-side, preferably not on the same system (use tow PCSs or VMs)
inspect each node output using the viewer, until you find where the two diverge.

Ivanohe73 · August 5, 2019, 10:13pm

Frederic,
I don’t see the warning in 2.3.14 on a centos 7 pc.
Thanks for the tips. I will post an update once I can confirm some of the things. So far I think there is 2 separate issues. If so, I will open bug report for each.

devernay · August 5, 2019, 11:56pm

You can see the warnings in ing the error log (Display->Show Project Error Log)
You can also use the environment variable NATRON_DISK_CACHE_PATH to get two versions of Natron running simultaneously but using different disk caches (it was not documented, I’ll add a section in the doc on environment variables)

Ivanohe73 · August 6, 2019, 3:05am

Ok thanks. I was able to compare the warning and found there is a new one. I open a bug for this issue : LensDistortion issue in Natron 2.3.15rc6 · Issue #416 · NatronGitHub/Natron · GitHub
I open issue Random freeze when rendering · Issue #417 · NatronGitHub/Natron · GitHub for the freeze rendering issue.

I will try to build intermediate version of Natron (from 2.3.14 to 2.3.15rc6) with a normal build and see if I can find the problematic commit. (assuming I can build easily older version of Natron)

devernay · August 6, 2019, 6:14am

The major change is when I fixed the the Python Gil handling

Ivanohe73 · August 22, 2019, 2:21am

While trying to fix an issue I am trying to build a complete tgz from my own repo of Natron and branch and got some error :

docker run -it --rm --mount src="$(pwd)/builds",target=/home/builds_archive,type=bind --env GIT_URL=https://github.com/Ivanohe73/Natron.git --env GIT_BRANCH=rotopaint natrongithub/natron-sdk:latest

./launchBuildMain.sh: line 306: GIT_NATRON: unbound variable

I tried adding a
--env GIT_NATRON=https://github.com/Ivanohe73/Natron.git

but got error:
Unknown plug-in repository URL https://github.com/Ivanohe73/Natron.git

I also forked all the plug-in in case and tried adding other options like GIT_URL_OPENFX_IO , etc but I still get the Unknown plug-in repository error…

rodlie · August 22, 2019, 1:24pm

github.com

NatronGitHub/Natron/blob/RB-2.3/tools/jenkins/launchBuildMain.sh#L203

    
      
          # By default breakpad is enabled
          if [ "${DISABLE_BREAKPAD:-}" != "1" ]; then
              DISABLE_BREAKPAD=0
          else
              echo "DISABLE_BREAKPAD specified: crash reporting support disabled"
          fi
          
          
# Remove any previous jenkins artifacts from previous builds
          if [ -n "${WORKSPACE:-}" ]; then
              if [ -d "${WORKSPACE}/jenkins_artifacts" ]; then
                  rm -rf "${WORKSPACE}/jenkins_artifacts" || true
              fi
          fi
          
          
# Do we need to build Natron ? No if the script was not invoked from a Natron CI project
          if [ -z "${GIT_URL_IS_NATRON:-}" ]; then
              GIT_URL_IS_NATRON=0
              isNatronRepo "$GIT_URL" && GIT_URL_IS_NATRON=1
          fi

github.com

NatronGitHub/Natron/blob/RB-2.3/tools/jenkins/gitRepositories.sh#L21

    
      
          # Natron is distributed in the hope that it will be useful,
          # but WITHOUT ANY WARRANTY; without even the implied warranty of
          # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
          # GNU General Public License for more details.
          #
          # You should have received a copy of the GNU General Public License
          # along with Natron.  If not, see <http://www.gnu.org/licenses/gpl-2.0.html>
          # ***** END LICENSE BLOCK *****
          
          

          
# List here the repositories
          GIT_URL_OPENFX_OPENCV_GITHUB=${GIT_URL_OPENFX_OPENCV_GITHUB:-https://github.com/NatronGitHub/openfx-opencv.git}
          GIT_URL_NATRON_GITHUB=${GIT_URL_NATRON_GITHUB:-https://github.com/NatronGitHub/Natron.git}
          GIT_URL_NATRON_GFORGE=${GIT_URL_NATRON_GFORGE:-git+ssh://natron-ci@scm.gforge.inria.fr/gitroot/natron/natron.git}
          GIT_URL_OPENFX_IO_GITHUB=${GIT_URL_OPENFX_IO_GITHUB:-https://github.com/NatronGitHub/openfx-io.git}
          GIT_URL_OPENFX_MISC_GITHUB=${GIT_URL_OPENFX_MISC_GITHUB:-https://github.com/NatronGitHub/openfx-misc.git}
          GIT_URL_OPENFX_ARENA_GITHUB=${GIT_URL_OPENFX_ARENA_GITHUB:-https://github.com/NatronGitHub/openfx-arena.git}
          GIT_URL_OPENFX_GMIC_GITHUB=${GIT_URL_OPENFX_GMIC_GITHUB:-https://github.com/NatronGitHub/openfx-gmic.git}
          GIT_URL_NATRON_TESTS_GITHUB=${GIT_URL_NATRON_TESTS_GITHUB:-https://github.com/NatronGitHub/Natron-Tests.git}
          
          
# List of known Natron repositories

Ivanohe73 · August 22, 2019, 5:36pm

Thanks @rodlie !

So here is the updated way with docker

to use you own git repo for Natron:

mkdir builds
docker run -it --rm --mount src="$(pwd)/builds",target=/home/builds_archive,type=bind --env GIT_NATRON=https://github.com/MyGitLogin/Natron.git --env GIT_URL=https://github.com/MyGitLogin/Natron.git --env GIT_BRANCH=mybranch natrongithub/natron-sdk:latest /bin/bash

sed -c -i "s/^GIT_URL_NATRON_GITHUB=.*/GIT_URL_NATRON_GITHUB=\$GIT_NATRON/" gitRepositories.sh
./launchBuildMain.sh

devernay · August 27, 2019, 11:33pm

Fixed in tools/jenkins: GIT_URL_IS_NATRON=1 can be used to launch a Natron bui… · NatronGitHub/Natron@a8cc6be · GitHub

If GIT_URL is an unknown Natron repository, GIT_URL_IS_NATRON=1 can be used to force a Natron build.

magdesign · January 2, 2020, 11:31am

Could you please give a short example on how to solve the: Unknown plug-in repository URL

Trying to compile a custom build with:

sudo docker run -it --rm --mount src="$(pwd)/builds",target=/home/builds_archive,type=bind --env GIT_NATRON=https://github.com/NatronGitHub/Natron.git --env GIT_URL=https://github.com/splidje/Natron.git --env GIT_BRANCH=#439 natrongithub/natron-sdk:latest /bin/bash
sed -c -i "s/^GIT_URL_NATRON_GITHUB=.*/GIT_URL_NATRON_GITHUB=\$GIT_NATRON/" gitRepositories.sh
./launchBuildMain.sh

Where would I add the GIT_URL_IS_NATRON=1 command?

rodlie · January 2, 2020, 11:45am

Add GIT_URL_IS_NATRON=1 together with the other env’s on the first line and remove the sed … line. That should work(?), note that I’m not a big docker user.

magdesign · January 2, 2020, 12:02pm

Thanks, using this command started the compiler (splidje build, will not compile):

sudo docker run -it --rm --mount src="$(pwd)/builds",target=/home/builds_archive,type=bind --env GIT_URL_IS_NATRON=1 --env GIT_URL=https://github.com/splidje/Natron.git --env GIT_BRANCH=#439 natrongithub/natron-sdk:latest

or rodlie build:

sudo docker run -it --rm --mount src=“$(pwd)/builds”,target=/home/builds_archive,type=bind --env GIT_URL_IS_NATRON=1 --env GIT_URL=GitHub - rodlie/Natron: Experimental Natron Repository --env GIT_BRANCH=SetDefaultProjectFormat natrongithub/natron-sdk:latest

bonalex01 · March 21, 2020, 6:05pm

Hi, i’d like to make the topic wider.
How do you edit Natron code ? What IDE do you use ? ( Eclipse, Visual Studio, QTCReator ?)

For a newbie like me it may be easier to copy someone else’s configuration rather than bothering the others with questions.

Alexandre.

rodlie · March 21, 2020, 6:31pm

IDE depends on what you prefer, I myself prefer to use QtCreator for most projects, especially for Qt based projects (like Natron). VSCode is also nice (not a IDE, but a good generic code editor)

Randalix · March 28, 2021, 8:07am

I’m sorry to ask stupid questions. Never used docker before. It seems like it compiled sucessfully with the commands above. But I don’t know how to launch it.
Can someone help? Links helping to understand are also much appreciated.

devernay · March 28, 2021, 2:58pm

you should find a Natron snapshot in the ‘builds‘ directory

Songtech-0912 · March 31, 2021, 1:43am

@devernay May I ask if you can hot-reload Natron using the docker-provided SDK? As currently this is only possible by building from sources manually (using qmake and make).

devernay · April 14, 2021, 12:43am

I don’t understand what “hot-reload” means.

Songtech-0912 · May 1, 2021, 4:11pm

Never mind, I found you can’t. What I had meant by “hot-reload” is like a live development server for web development, where you can open localhost and see that changes live as you made them in your text editor. I thought you could do the same for Natron, but it seems not.