Backing up RT's PP3 files - is there a python doctor in the house?

Software RawTherapee
,
1

I few weeks ago I decided I wanted to learn some programming and started teaching myself python. For one of my first projects, I decided to write a backup script for my RawTherapee pp3 files. I wanted to solve 2 shortcomings with how RT manages its pp3s:

  • There’s no way to save different edit snapshots in the pp3 itself
  • Backwards-compatibility issues over different RT versions

The script starts in a folder and will recurse into all of its subfolders. It looks for .PP3 files in the directory and will make a “PP3 backups” directory if it finds them. It will then copy all the pp3 files into the backup directory, and append the RT version number and an incremental counter to the filename.

If you run the script again, it will examine all the PP3s it finds, and compare them to the pp3s inside the backup directory. If they are different, it will copy the newer version of the pp3 without changing the old one.

The idea is that you can experiment with different “looks” and not have to manually save & organize all the different pp3s.

There is a bug in my script that I cannot figure out. If anyone here is fluent in python, maybe they can tell me what the problem is?

I’m attaching the script and some example pp3s to this post. There are problems with the pp3s named “Will Coffin.jpg.pp3.” There is something about those files that when I run the script, they will always copy and overwrite the existing backups. The regex search at line 60 returns None when it should be finding a match in the backup directory. So far, all the other pp3s I’ve tested it on work fine.

You can test the script on other pp3s by opening one in a text editor, changing it somehow, and saving. The next time you run the script, it will detect the change and copy that version to the backup directory.

If you want to test the script, you’ll have to change line 13 to point to the path where your test pp3 files are. Right now, I would not feel safe pointing it at pp3 files that you care about.

I know there’s a lot of technical knowledge on these forums, so figured this would be a good place to ask.

Edit: I put the code on pastebin if anyone wants to glance at it but not download the zip file.

pp3 Backup Script.zip (184.1 KB)

2

I can’t provide a solution, but can provide this old programmers proverb: I had one problem and solved it with regex, and now I have two problems.

1 Like
3

Maybe the problem is the space in the filename?

4

Using emacs or vi???

5

LOL as someone who just learned a bit of regex, I can definitely appreciate that!

6

I thought that may be causing it, so I added a space in the name of another pp3 file to see if it would break it but it still worked fine. I should try removing the spaces from my problem files to see if it solves the problem, in which case I can probably modify the script to remove spaces from filenames, but that seems kludgy. I’d do it as a last resort.

I’m hoping there’s some issue I can tweak out of my regex.

7

I haven’t looked at your code, but I also wrote my own file renamer, and I didn’t need any regexes. What I do is, for every file, separate the name from the extension (so “image_01.pp3” becomes “image_01” and “pp3”). I’m pretty sure Python supports this. You can further split the name based, say, on underscores (so “image_01” becomes “image” and “01”). With this approach, I found that comparing files and adding counters becomes… not really easy, but less tricky. HTH.

8

They are saved in ART, ie the snapshots. You might be able to look at that code or ask @agriggio about the implementation…

9

I think it is because you are using the re.VERBOSE flag. This will cause all whitespace to be ignored (including in the stem part). If you rewrite the re.compile() call without all the linefeeds, spaces and comments and then omit the re.VERBOSE flag it should work.

At the moment, I can’t of a way to pre-process the stem part so that you could use it with re.VERBOSE.

1 Like
10

Holy crap that is it! It worked! I can’t tell you how long I beat my head against the wall trying to figure this out.

The good news is that after struggling for hours, I am way more familiar with Regex, string interpretation, etc… maybe that practice will pay off in the future.

2 Likes
11

In my elation for you helping me with this, I just sent pixls.us $30 and told them it was from you. :innocent:

Thank you also for your post. You made me realize that I did get stuck in my own head and fixated on the regex thing. I was actually in the process of rewriting those parts of the code to cut down on the regexes but then RichardT figured out the source of my problem.

Not related, but I also find it mildly amusing that Regex is a homonym for “Rejects.”

1 Like