GUI overhaul with QT 5.12

I’ve pushed my work on my own fork on github: https://github.com/yomgui1/Natron

THIS IS ONLY FOR YOUR EYES
I mean…

  • no documentation on what I’ve changed
  • no documentation on how to build/use
  • only build on my machine ™
  • only for me ™
  • only to store my work somewhere else than on my HD.
2 Likes

Hi @yomgui
Thank you for that work, but be careful to keep backward compatibility with Qt4 as far as possible (use #ifdefs), avoid useless changes (siuch as reindenting source or changing the file order in a Makefile), and break down all your changes as much as possible, so that they can be cherry-picked. This kind of commit can not be merged upstream:
Showing 349 changed files with 49,540 additions and 40,607 deletions .

All commits still have to be reviewed before integration, so try to make the reviewer’s work as easy as possible.

Ahhh I’ve forget DO NOT MERGE point at the end ! :sweat:

@devernay about Qt4/Qt5 compatibility: big warning!

Sure Qt API itself is «almost» compatible, but the issue is on PySide.
PySide2 typesystem files and generated code is not compatible at all.
I mean the wrappers generated by PySide v1 must be keep away of PySide2 ones.
And the overall code may requires some changes for both of them.

The current way to record this generated code in the GIT repository cannot work.
So forget to make the code compatible at this layer.

Sure, I’ve not tried too long if there is a way to do that. But it’s out of scope of what I can do in my spare time.
I know what you’re thinking and what you hope, but if the effort to keep this compatibility is too high, no one would participate on maintenance.

When I’ll do push request, be sure that I’ll keep them as atomic and clean as possible.
This branch is only for me.

[Edit1]
When writing this message I’ve think that using a separated branch for Qt4 and Qt5 is the best way.
Qt4 branch can continue in «maintenance» mode (with Qt5 patches backport when possible).

[Edit2]
@devernay I’m testing a Natron sources tree configuration to have a cohabitation of Qt4 and Qt5.
IMHO, it’s a bit ugly and force to build and maintain two projects. But we can have Qt4 and Qt5 in the same GIT branch in this way.
I’m going to make a build procedure, as I’m introduce docker in the build system (but I’m using jenkins scripts).
When it’s ready for review, a PR will be done.

2 Likes

What ? :roll_eyes:
Why QT4 with QT5 ?

@razorsharp Not at runtime for sure.
I think @devernay want to maintain a Qt4 branch to see if a bug is introduced only by Qt5 and having a «production proof» Natron (whatever this really mean).

BTW, I’ve joined a discord channel if you want to exchange some ideas with me : https://discordapp.com/invite/TP9GT3e

The existing code was already compiling with Qt4 and Qt5, what’s the point in removing Qt4 support, when all releases were built and tested on Qt4? Did you even run the full set of unit tests on the Qt5 version?

I agree to have a different set of shiboken-generated files for Qt5 (in another directory rather than a separate branch), but all other source files should compile with Qt4 and Qt5. The difference is not enough to justify a separate branch for now.

And, most of all, what’s the point in focusing on the upgrade to Qt5, which will bring its lot of issues, rather than fixing the existing issues? What’s the added value for the customer? Is it more important to have a compositing version that uses the last version of Qt, or to have something that is production-ready?

1 Like

It might be part of attracting new developers. It’s not really a point in a projects favor if it is built using a UI toolkit that’s been unsupported for a while. On a more practical side many distros are pushing to drop (or already have) qt4 from their repos (I know you build it from scratch for your binary releases, but there are people that prefer source/distro-packaged releases). These are not points for dropping qt4, that’s a different discussion, but reasons why I believe it to be very valid to try and get a stable version of natron with qt5.

1 Like

Blockquote
Is it more important to have a compositing version that uses the last version of Qt, or to have something that is production-ready?

lol, I’ve so much seen this argument during my work in industrial domain…
Easy, none of them: https://en.wikipedia.org/wiki/False_dilemma

First production-ready is not a “simple definition” that anyone are agree with its bound. This is a per-project and per-people definition. We need to stop to use it as excuse to anything.
About Qt4: do you think that using Qt4 is production aware if I need to say to any team that the software they use is deprecated by those who made it and no bug-fixes will be released anymore?
They will get a smile :slight_smile:

Sure they don’t care about Qt4 vs Qt5, they want a software that can be used everyday and works without knowing how to install it.

BUT…

To make this last sentence a reality, we (developers) need to have right tools.
This is our requirement. We don’t sell Qt (we even not sell anything in fact).
But we need something strong at development level, to make things strong at user level.
It’s a well known causal effects.

For the fun, this is a real story.
I’ve work for a customer last year (world big one in TV broadcasting domain) that has exactly followed the gimmick “no fancy dev tool, no upgrade, don’t touch working code, only customer requests matter”.
Today all the code is java 5, tons of deprecated libraries with known bugs, running on a custom and manually maintained gentoo.
After 10 years of existence they decided to change for an Ubuntu has the tech lead of the
project and developers were upset to maintain libraries in Gentoo.
But it was too late, most of senior dev and architect have leaved the project as they were upset to never follow the usual technology improvement (not the only one reason for sure, but all ones have given this one).
Consequences: only 1 senior SW remains on the project, nobody understand the code (+1M lines).
But customers are happy with the SW in fact! … except that now they will not get much support for it as the team is reduced to the strict minimum to live and improvements are made at high cost.
Only because managers don’t care about developers.

«Happy Developers Make Better Software»
«No Devs, No Software»

Sorry @devernay if you think I’m a bit harsh with you, nothing personal, but you need to understand that Natron is an opensource software that needs developers first or the project will die. No ones is going to help you in the FOSS world if the technology behind doesn’t appeal them.
You and Alexandre have made a fantastic job: give it now a good and long life.

PS: yes, I’ve run the tests… all goods excepts 3 ones due to non-built pluggins. As this first stage was only a proof of concept, I’m happy with that

1 Like

And Now for Something Completely Different

As for me Qt5 migration is a reality and as I’ve made an effort to keep Qt4 code working
with Qt5 one, these are my goals for next weeks:

  1. Continue to add one by one 3rd party libraries into the new Docker based build system.
  2. Be able to build Natron (not plugins) with a CY2019 compliant SDK (build on CentOS 7) and check if it runs on recent platforms likes Fedora 29 or 30 and Ubuntu 18.04.
  3. Write a documentation on how to build Natron with the new Docker system.
  4. Start to add plugins to the build system.

Notice that with Docker the build system is now fully independent of the host build machine and where is located. Nothing to install manually: only few commands to copy/past into you terminal. Fully automatized.

[EDIT1]
Step 1 and 2 done (but tested only on a Fedora 29).
This is the patch on my qt5-dev branch here: https://github.com/yomgui1/Natron/commit/b2e931d2959549fc0cb380f0d2597815c915f4df
There are some quick instructions on how to build it into the commit msg.
If some people are courageous enough to try them and report me issues on the build system, thanks a lot on the help.
Please note this is done on my fork of the genuine repository.

@devernay for sure, this patch is not ready for an official PR: I need to separate Qt5 port from the new build system, as the second part absolutely not giving release build and not gives all features.
But you can keep an eye on it :wink:

[Edit 2]

I’ve broken the Qt4 build in my previous commit.
This is a fix, comes with instruction to build Qt4 version also with Docker: https://github.com/yomgui1/Natron/commit/18804bb9d15324cb1247b88fef5f232f0c1577f2

Time for a game: spot the differences in image below. Which one is Qt4? Which one is Qt5?

4 Likes

Good to see your progress.
Here i’m not able to build ( due to lack of experience with docker).

Well in fact docker.com has all of them :smiley:

For CentOS https://docs.docker.com/install/linux/docker-ce/centos/
For Ubuntu : https://docs.docker.com/install/linux/docker-ce/ubuntu/
For Fedora : https://docs.docker.com/install/linux/docker-ce/fedora/
For Debian : https://docs.docker.com/install/linux/docker-ce/debian/

1 Like

FYI The docker image with the full Natron SDK and all the patches is almost ready. And it is based on CentOS6 for maximum compatibility. Many packages require patches that are sometimes essential to use with Natron (but these patches are more and more integrated upstream, eg https://github.com/ImageMagick/ImageMagick/issues/1488)

@devernay Noted! Thanks
I’ve not yet reach this dependency (ImageMagick), until now, I only build Natron core itself, no plugins. But I’m able to use pre-build ones from your released archives.
talking docker image, did you see my last patch with a CentOS6 dockerfile that uses your genuine scripts to build Natron?

no, I didn’t see any PR.
The build script now generates a Dockerfile, see https://github.com/NatronGitHub/Natron/tree/RB-2.3/tools/docker/natron-sdk

@devernay I see you are working on RB-2.3 branch. I base my work on master…

Don’t use master. RB-2.3 is the default branch for very good reasons. The engine in master is awfully buggy.

@devernay my god, why I’ve you not said that before? I’ve even read into an article with interview of you and Alexandre that contributors have to use master…
And I’ve seen also you rewrite all the linux build script separating all deps rules. That’s an excellent job, I regret to not been aware of that before. I feel like doing my work for nothing.
We should better communicate on such things to stop work duplication.

@devernay I’ve tried your docker build script. After a while (I went to sleep… so 6 hours later), I’ve seen it stuck on pcre build, no information explained. I only can press CTRL-C to exit. There are not possibility of post-mortem debug.

This event shows a big issue with the whole process: don’t build SDK in the “docker build” stage. It takes a long time to do and in case of error, all the process should be restarted. This implies full download of all Natron deps archives and rebuild all successful steps as these are not cached by docker.

What I can propose is to take a look to my Dockerfile: only OS packaging is done by the Dockerfile, then I’m using docker volume to backup tarballs, SDK, build dir, and so on.
This way in case of any issues, tarballs and already built stuff remain into the volumes.
Just have to restart the build command and the build restart where it stopped.

Note: the “checkpoint” trick is just a workaround, it don’t help more if we need to enter into a shell to try various commands. Only developing into a docker instance can help

I don’t know what interview you are referring to, but that was probably before the master branch was broken and Alexandre left without fixing it.
The docker build now has several checkpoints, and you can easily add more. If a package fails to build, the log file for that build is output.
Once again: to build standalone binaries, you depend on libraries from the distrib. You can use that to develop Natron, but not to distribute portable binaries.
The current docker sdk build is in the RB-2.3-sdk-upgrade branch.
I will push the SDK image to dockerhub when it’s ready.