Parsing PP3 Sidecar Files

timelapse
rawtherapee

(Lukas Stratmann) #1

Hi!

I would like to parse PP3 files in another program. Is there either a general way to determine the type of the value for a given key, or is it maybe possible to externally use RawTherapee’s own function for parsing PP3 files? So far I haven’t been able to find the relevant sections in the sources, unfortunately.

Some background: Years ago I used Johannes Bildstein’s Lapsestudio for deflickering time lapse sequences. A unique feature of Lapsestudio is that it lets you define a number of PP3 files as keyframes and it will then interpolate all key-value pairs while also removing flickers. Unfortunately, parsing PP3 files appears to be solved in this case by determining the data type of each key-value pair line by line, hard-coded. Johannes adapted this method for PP3 versions up to 321, newer versions don’t work anymore. So I’m hoping that there might be a way to kind of future-proof his software.
Lapsestudio can be found here: https://code.google.com/archive/p/lapsestudio/ (There are also a few exports of this project on Github.)


(Ingo Weyrich) #2

the relevant sections in the sources


(Lukas Stratmann) #3

Thanks! So it looks like if I wanted to create something like PP4 with integrated data types*, all I’d have to do for loading is to prepare the data for Glib::KeyFile.load_from_data, and saving shouldn’t be too complicated either as saving doesn’t need to be backwards compatible… How likely would such a change be accepted on the master branch?
(I haven’t yet contributed to other people’s open-source projects, in case that wasn’t obvious, but I’d like to give it a try.)

*) Maybe instead of switching from the INI format to XML or YAML it would already be sufficient to force floating point values to include a decimal point, strings to be surrounded by quotation marks (to make them distinguishable from booleans), and maybe to enclose double lists for curve data with double_list(0;0;1;), for example.


(Ingo Weyrich) #4

I really don’t know about the likelyhood to accept a change before we saw it.
Please open an issue at https://github.com/Beep6581/RawTherapee/

Ingo


(Peter Lavender) #5

Wow, the things you come across reading through the forums.

Does this software work like the LRTimeLapse for the Lightroom users?

When I need to deflicker, I’ve been using the Deflicker.pl perl script that does the job reasonably well, but I’ve always looked at the LRTimeLapse as something pretty handy for the task.


(Lukas Stratmann) #6

I think it does, although I can’t say for sure since I’ve never used LRTimelapse (it’s Windows/Mac only, unfortunately).


(Peter Lavender) #7

I’ve spent the night so far looking at the way LapseStudio parses the PP3. So far it would seem it’s just a case of adding in the missing configurations items to the new PP3.

In saying that, the PP3 seems like a fairly standard INI file format, so I’m not sure why it can’t be read with a general INI file class… and it’s been ages since I’ve done anything like this, but I’d have thought that C# would have one of them.

Hmm… OK, seems maybe it doesn’t: https://stackoverflow.com/questions/217902/reading-writing-an-ini-file

I’ll have a bit more a play with this over the next few nights as I’ll crack this one way or the other. :slight_smile:


(Peter Lavender) #8

Was there ever any doco written for this application?

I"m not sure how the key files are meant to work and what PP3 you should be using…


(Lukas Stratmann) #9

I agree something should probably be done there. It seems like the current method relies on key-value pairs always being at the same position within PP3 files. But I think the bigger problem is that LapseStudio has to rely on outside knowledge about data types due to the PP3 format.

Not that I know of, but when it works it’s pretty straight-forward to use (provided you can read the tool tips, which for me only show up as white boxes):

  • Go to Settings, select an appropriate number of threads to use. Set “Used Program” to “RawTherapee”.
  • Click on plus icon (add images)
    • Select folder containing image sequence
  • Wait for images to load
  • Click on “KF” check box for each image you want to mark as a key frame
    • Select the pp3 file of the edited key frame image
  • Click on the cogwheels icon (“Calculate Brightnes”), wait(?)
  • Click on the save icon (“Process Frames”, downward pointing arrow), wait(?)

Current progress on Github is a bit hard to track as there is no main project. However, Arttu Voutilainen appears to have done the most recent work on his WIP branch (I just found out that he’s already added support for PP3 version 326, but it’s still only a temporary solution); otherwise the most current version of the project I found is Vishal Prabhu’s export of the Google Code project (not yet with support for PP3 326).


(Peter Lavender) #10

Thanks, good to know.

I’ve been working on getting 326 working… would have preferred finding someone else had done it. :slight_smile:

Thanks for the run down on the way to do it, I wasn’t sure if the images were meant to be pre-processed. Makes sense that they are. I guess the key files are points in the series where the changes in light have some significant effect? IE, what files do you pick as a key file… and while I was that I’m about to go google it, and pull the updated git repo.


(Peter Lavender) #11

OK, after finally merging the branch, it didn’t work for me. Complained about the boolean datatype from memory.

My version while, successfully reading in the new file format crashes the internals of the application. Debugging that will be a bit of a chore as I’m so rusty with coding. I’ll keep working through it as I find time.


(Lukas Stratmann) #12

Good question. I’m not sure it’s the best way to go about it, but this is what I used to do:

  1. Edit the first image of the sequence in RawTherapee.
  • In the file browser, copy the processing profile from the first image to the remainder of the sequence.
  • Scroll through the image sequence until you find images that you think should look different. (E.g. WB might be off after the sun has set.)
  • Make changes to this image (or an image a few frames before or after).
  • Either write down the name of the images you changed or copy their PP3 files to a separate folder in order to make selecting the key files in LapseStudio a bit easier.
  • Repeat from step 2. until you’ve reached the end of the sequence.

Do you mean you’re working on your own solution or are you trying to get Arttu’s version to work? I haven’t tried his yet…

I started working on how RawTherapee saves PP3 files, but I’m afraid I might have to delay work until after my exams.


(Peter Lavender) #13

I have my own 326 class for parsing the new file version, which reading the PP3 it works fine. I made a few mistakes and actually haven’t tried doing a complete test run. I’ll see how it goes tonight.


(Peter Lavender) #14

Spent a bit of time on this tonight. Appears I’m just down to working out the right data values to write out to the PP3. Currently stuck with a curve. Not something I’m going to go much further with tonight.