Write Rawtherapee output of image sequence to video clip

Hello, I’ve been using Rawtherapee command line for processing DNG based raw video, exporting to 16 bit tiff, then using FFMPEG to transcode to Prores. I just recently upgraded to a BMPCC4k, and the tiff intermediary file size is problematically huge with this workflow, as a 12 minute 4k clip produces enough tiff to run out of SSD storage space.

Anyone have any workflow ideas of how to send the file output of Rawtherapee directly into FFMPEG for encoding in Prores, without saving a 16 tif intermediary?

Edit for clarity

I’m not sure if this works, but ffmpeg has the option to take pipe input. https://ffmpeg.org/ffmpeg-protocols.html#pipe
Maybe you can use this in conjunction with rawtherapee-cli to pipe the tiff output directly to ffmpeg? https://rawpedia.rawtherapee.com/Command-Line_Options

I poured over both, and tried several different commands, but nothing worked.

I would try batching a directory into mentioned pipe feature: rawtherapee-cli -t -c directory | ffmpeg -f image2 -i pipe -c:v prores_ks -profile:v 5 out.mkv

How many TB does it take to juggle 12 minutes of Prores tiffs? 6TB?

2 Likes

Thanks, 700 Gb for just 9 minutes of 24p 4k tiff.

No luck with that syntax or any other ffmpeg syntax for the pipe, referring to the manual.
I have been getting nothing but errors with the ffmpeg output
It seems that Rawtherapee isn’t throwing anything into the pipe, as when I run
rawtherapee-cli.exe -t -c test.dng | %magick% - -resize "50%" magic_out.tif

I get:
test.tif already exists: use -Y option to overwrite. This image has been skipped. magick.exe: no decode delegate for this image format ’ @ error/constitute.c/ReadImage/572.
Also, the tiff just gets written to disk by Rawtherapee anyway, and not passed on to anything.

AFAIK, there’s no common log facility in RT that could be muted, only printf()s (and friends) here and there. So even if there was a possiblity to stream out data to stdout, it would be corrupted by occasional status messages.

Best,
Flössie

Thanks, I guess that issue was what was needed. I’m excited for the creative workflows this will enable.

@Waveluke I hope you did see the date when the issue was opened …

OOF. Would making a comment on Github bump visibility?

It does if you sort issues by recently-updated:

@Waveluke @Hiram … and I get an e-mail every time an issue is updated, so that at least gets a mental note in my brain that somebody is still interested in a feature. Maybe other devs get this as well.
At the very least this means that whenever some overactive programmer suddenly wants to join the team, I could point them to a few useful issues to take up :wink:

Hmm, maybe a named pipe or something?

Hi @Entropy512, there is now a branch on GitHub that tests diverting all console output to stderr. This ought to get a clean and usable output on stdout to pipe through to other places. If you’re interested, feel free to tag along in issue Flössie mentioned before.

OK, I’ll try to take a look this weekend. One thing that I’m not sure about is how ffmpeg will handle a sequence of TIFFs in a pipe - whether or not it needs some additional framing/containerization or if it can differentiate the individual TIFFs in the pipe.

It should take each frame in until you hit control-c. If your output is to be a seekable format, it won’t work right.