I have a question on this issue. I always wondered why Natron has trouble with H.264/5 codecs. How is it handling things differently then, let’s say Kdenlive. Almost all of FOSS uses FFMpeg. Just curious what the difference is. (I know Natron is a professional application and designed for professional codecs not for working with mainly end user codecs)
Thats because with many video codecs, frames are made to be encoded or decoded sequentially (the only exceptions are codecs using only intra frames, such as ProRes or DHxHR).
Natron accesses the frames in random order, which make things a bit complicated, especially using the FFmpeg API which is not made for that and very low-level. If we could have a robust API over FFmpeg to just “fetch fram f of that video sequence”, that would solve eveything.
Of course, rewriting properly the FFmpeg reader would also solve this, but the FFmpeg API is a mess, and a moving target.
For writing videos, Natron should be able to do it properly, but there are still a few incompatibility issues (eg with DNxHR) that have to be fixed. However, I never recommend rendering a complex graph to a video.
I think if there’s a clear section in the manual about how to use other FOSS tools to perform these tasks, this will help a lot.
I’m asking for this doc from you, because we have no developers to do the code rewrite, but we have talented users who know solutions and can document these, even if they use external tools.
The doc should basically be text and images. Preferred text format is markdown or rst, but it’s ok even if it’s a word doc, as long as you don’t use a crazy layout. Whantever the format is, I’ll convert it to RST first using pandoc, and then edit
Thank you for the information. It cleared up my lack of understanding with how Natron works with H.264/5 I get the rest of things. I’ll try and help CGVirus get something together.
The problem occurs with the way H264 compression algorithm is designed. To encode, h264/5 uses b and p frames. Suppose a 1 second shot of 30 fps has 30 frames. h264/5 will pack similar pixels of some forward frames (B frames) like frame no 2 and 3 in frame no 1 to compress the size. Also it uses no 3 and 4 to compress similar pixel in no 5 frame (p frame). Then blend them all together to create final frames. (less frames in packets)
That’s how it’s smaller in size.
Also to decode it uses again these algorithm for which CPU needs more power and time just to decode and extract those frames again.
Now this is the problem. First the decoder needs to go back and forth to decode these frame. Secondly, we need to capture the frames almost in real time and send them to memory. Waiting for this B and P frame to process will kill the header to scrub and for compositing will create a latency. This latency will record in the memory. That’s why we will have frame jumps. Which will also halt other nodes in latency. This will become a problem for trackers and roto. As they will miss some frame in screen pixels. So the roto and tracker will be inaccurate.
In VFX and editing, if the farm wants to (usually and for sure never) use h264/5 as input, they can use Intra frames. It’s easy to do in ffmpeg using arguments: -bf 0 and -g 1. Which ensure h264 will not use b and p frames. But h264 decoder will run the cpu processing anyway and although we will have all frames in memory, the ammount of processing power needed for decoding just the video is not even close to effective.
Usually in editing Intermediate format like proress or dnxHD or QTL is used. Because they don’t use any aggressive frame binding rather packs bunch of still images in a container with quality loss or no loss.
Usually for vfx image sequences (PNG/TIFF/TGA/EXR) or above Intermediate format are used because it’s simple to decode and send to memory immediately. And better for tracker and roto as well.
The basic code for Natron to request FFMpeg execution for image transcoding is simple enough but should it be encapsulated in a node or should it be a Menu wizard script? Verified Natron script code below.
import os
import subprocess
os.chdir(’/home/tom/Videos/test/’)
subprocess.call([‘ffmpeg’, ‘-i’, ‘test.mov’, ‘image%d.jpg’])
Should this be a seperate node, or should the read node have an option to transcode to image sequence (maybe even by default) in a subfolder adjacent to the original file?
The former might/should/could make the read-node complain about non-image sequences. The latter would need a logic inside the read node to opt for the image sequence once one is there.
I think that the best solution is a wizard from the main menu. It would avoid any confusion with existing nodes and is the way other programs like Kadenlive do it. Creating it as a separate node or a menu wizard also removes it from the intended functionality of the read node IMO. I’m open to any ideas though.
I haven’t tried coding a script with pop up windows yet in Natron but from the docs it looks possible.
Programming question: How do I write the code for the on click event for the buttons?
Programming question: How do I write the code for the on click event for the buttons?
You can see the audio_vlc.py line 13-16 & 166-171 (linking)
You will need a your_main_filenameExt.py for external execution
see Audio_VLCExt.pyfor the implementation. line 54- 99
There’s some questions I don’t know the answer to, like which format to use for 10/12 bit input and there’s no doubt more that needs to be added to the ffmpeg commands.
Also if anyone knows Windows and Mac software and can write about it please do!