Processing profile compatibility between different versions of RT

It looks like maybe I was dreaming. A little experiment produced this report which shows the differences in the processing profiles produced by RT Version 5.3 & RT Version 5.4. In this case the same raw file is opened by both RT 5.3 & RT 5.4 for the first time (i.e., no prior profile existed) and RT was terminated without doing anything.

Absent some compelling explanation to the contrary I must conclude that these profiles are version specific and should not be used with a different version of RT. The really bad news is that RT doesn’t even alert the user to the discrepancy. Even when the user does nothing to an image open for editing, it appears as though RT updates the profile created by a different version of RT, which I think means that RT just created a compatibility problem, likely unbeknown to the user, in the sense that a valuable profile that produces desired results using another version of RT may no longer do so. Of course an insidious aspect of this is that the differences can be somewhat slight and difficult to recognize but if I’m not mistaken it is the desire for control over these slight differences that are a prime motivator for working with raw images in the first place.

While I’m very much a novice when it comes to raw photo development I do know enough about software development to know that this is neither good nor common practice. Therefore, my current suggestion would be that RT ought to minimally alert the user & request permission before altering a profile created by a different version of RT.

I may be a bit more sensitive than some to this issue because I was using the portableapps version of RT at one time. In that case, the portableapp platform/launcher was choosing a different version of RT based on the architecture (32 vs. 64 bit) of the computer on which it was being run. I happened to be switching back and forth without knowing much of anything about processing profiles. It did cause me some real problems and I have advised the portableapps folks that they ought to make some changes that minimally make the user aware when different versions are being invoked. I suppose if my suggestion, herein, to RT were accepted it would be a big help to the portableapp problem as well.

Example 1:

[Local Contrast]
Enabled=false
Radius=80
Amount=0.20000000000000001
Darkness=1
Lightness=1

Local contrast is a new tool in 5.4. Of course it needs settings in pp3 file. Of course these settings are not read by older versions of RT which don’t have this tool

Example 2:

[Channel Mixer]
Enabled=false

In RT 5.4 a lot of tools got an on/off button which wasn’t there in older versions of RT.

Example 3:
LevMethod=4_ => LevMethod=4

That’s handled correctly in RT 5.4. Means if you apply a pp3 written by a RT version < 5.4, RT detects this, applies the value correctly and writes the new vale 4 to the new pp3

Example 4:
FlatFieldClipControl=false => FlatFieldClipControl=0

That’s a bugfix.

If you would have had a profile before your comparison would look differently. When you open an image which does not have a profile so far, a default is applied. And this default is of course saved as profile together with your raw file.

So in my opinion, your report is (to some extend) comparing apples an pears.

OK! I did try to keep it simple with the idea that anybody, who cared, could easily duplicate the experiment.

I’ve now taken this same raw file which, BTW, took me quite a long time to render into something that I liked using RT 5.3. I then opened it using RT 5.4. The look without doing anything, which happened to be fresh in my mind for this instance, was dramatically changed for the worse. Now suppose after having converted from 5.3 and been using 5.4 for a long time I had done what I just did. I would likely be disappointed by the new appearance but may not recognize that it was changed from something I liked to something I dislike. Furthermore, the profile that used to work nicely in 5.3 has now been changed so that is no longer the case. In that, RT 5.4 ruined a profile that produced good results on RT 5.3 and that cost a lot of time and effort to produce. Also, RT 5.3 does open the profile created by RT 5.4 with no indication of this discrepancy even though the very first parameter contained in the profile specifies the RT Version.

What is the reason for tagging the profile with version information if not to alert users about such discrepancies?

This little experiment pretty much tells me that the profiles are NOT interchangeable in this case (i.e., 5.3 vs 5.4). While it seems possible that some other migration to a newer version may not be as bad, my basic conclusion is that one better develop procedures that prevent this from happening. It is hard to see how warnings from RT could do any harm.

I doubt that it means anything but just in case here is the profile comparison for this alternative experiment.

When it comes to the apple & pears analogy it looks to me like it is RT which is converting good apples into inedible pears.

Please provide an example (raw file and pp3 file) so we can try to reproduce.

1 Like

Hi,

In your 5.4 profile you have “histogram matching” enabled. That means that either:

  1. it was enabled without your intervention. In that case this is a bug: it would be great if you could provide more info so we can try to see what is going on; or
  2. you clicked on “auto-matched tone curve” in the exposure module, which, as expected, changed a bunch of parameters (exactly those that are highlighted in your diff). In that case, there’s nothing wrong, everything works as expected.

HTH

2 Likes

Being less destructive in your criticism would also help. Examples:

If you apply your pp3 file from RT 5.3, which cost you lot of time and effort to make, the right way (as full profile), you should get the same result in 5.4. If not, it’s a bug, which should be reported, or it’s caused by fixing a bug.

We take every issue serious. Of course, RT 5.3 can’t use tools, which are introduced in RT 5.4.
But going backwards is kind of low in our list of priorities.

1 Like

I used the provided 5.3 version pp3. I did following experiment

1- open a CR2 file with this pp3 in RT5.3, save jpg, close 5.3
2- open the same file with same pp3 in RT4, save jpg, close 5.4

Now the pp3 is a 5.4 version
3- open the same file with pp3 in RT5.3, save jpg, close 5.3

display the 3 jpg: look the same. I did’nt made further comparison in GIMP.

1 Like

My apologies! I’m really not trying to slam (put down) RT. The reason I’m spending my time doing this is that RT looks like a good & powerful tool to me. I’m still very much in learn mode and quite receptive to being shown where any of my findings or just plan interpretations are wrong. I’m new enough to raw image processing that I’m still trying to develop a workflow that might pass as good to a knowledgeable and experienced user. I think this matter of “Processing Profile” compatibility is an important consideration.

I also have a great deal of appreciation for open source software and recognize the enormous complexity associated with developing something like RT in such a manner. As such I am most grateful to all of the people doing this work.

My initial question was aimed at determining whether or not it is intended that processing profiles created by prior versions of RT can (and maybe it would be better to say “should”) be used with newer versions. Based on discussion herein it sounds like the answer is that such is intended. However, this discussion has also caused me to do some experimenting and based on that I remain a bit skeptical about the idea that such should be done. I’m thinking it might be possible that when it is desired to further refine the development of an image that was previously developed with an older version of RT that using an image file (i.e., in a standard format) such as an exported tif might be better. When it comes to workflow the idea being that once a desirable image has been developed it needs to be exported to a standard format.

I’m pretty sure that when it comes to forward compatibility it’s one thing to migrate from one version to the next but as the number of intermediate versions of RT increase the prospects for this to work well diminish. In that, for now I shouldn’t be discarding my 5.0 version of RT.

It hasn’t come up but I am using Windows and have followed these instructions for running RT in portable mode. This means I can easily invoke any of the 3 versions I’ve now used.

1 Like

See “Compatibility”
http://rawpedia.rawtherapee.com/Sidecar_Files_-_Processing_Profiles#Compatibility

1 Like

In your 5.4 profile you have “histogram matching” enabled. That means that either:

I had no knowledge of Histogram Matching. The answer must be “unintended”.

Something I did notice is that when opened in 5.4 there was a “tone curve” but while I had experimented with this tool in 5.3 I ended up not making any such change. When reopened in 5.3 the tone curve was still there.

Overall my edits were not very complicated. The reason I spent a lot of time on it is that I’m a novice and there was a lot of “do over” involved. I made some changes using both Highlight and Shadow compression to limit clipping that occurred in both the highlights and shadows. What seemed to really improve the image was done with the Saturation slider.

No need to apologize :sunny:

Yes, they can be used. In case the output of an old profile (e.g. from 5.3) is different when applied using a new version (e.g. 5.4) that can be caused by a bug, but also by a bugfix. In rare cases it can also be intentionally, but we try to get newer versions compatible with older pp3 files.

We don’t do this the other way around: Applying a profile from RT 5.4 likely will not give the same result in RT 5.3

As @Morgan_Hardwood pointed out here, there is already an issue to improve the case that you apply a profile which is not comatible (fails). But imho the first RT version which will solve this, will be > 5.6, because we can’t solve it for 5.4 by nature (as 5.4 is already released) and solving it in 5.5 would mean it would be solved for 5.6.

OK, I also did this using the raw file that is the subject of my experiments. Here is a link to an album with the 3 resulting images.

  1. Image1RT53 : Is the one a like
  2. Image2RT54 : What resulted from processing with RT 5.4
  3. Image3RT53 : What results from using profile created by by RT 5.4 in RT5.3
    An observation that I make is that RT 5.4 has something which I think is new called an “Auto Matched Tone Curve” (AMTC). I fiddled with tone curves when editing this image in RT 5.3 but didn’t think they helped so I reset the tool to default. In that, Image1RT53 did not have a customized tone curve. However, RT 5.4 did create the AMTC which appears to be a substitute for Tone Curve 1 even though I did nothing. The image in the film strip when I first launched RT 5.4 looked like Image1RT53. As soon as I selected it for editing it looked like Image2RT54.

Hi,

if you are sure about this, then please tell us more about the steps to reproduce. I.e., start from the 5.3 pp3, open it in 5.4, and check whether the “auto matched tone curve” button is checked or not. If yes, please upload the raw and the (5.3) pp3 here so that we can have a look. Thanks!

1 Like

OK! Not sure what I can tell you but the raw file and what I think is the good processing profile when opened with RT 5.3 are here →
EOS Rebel T60079.CR2.pp3 (10.8 KB)
EOS Rebel T60079.CR2 (23.2 MB)
To be sure, I’m a novice when it comes to digital image editing so I only used a few of the basic tools. Nothing elaborate or exotic. Likely only a few of the ones on the exposure tab. As I said I diddled with the Tone Curves but concluded that they didn’t help and ended up with what I’d say is either nothing or the standard linear curve.

EOS Rebel T60079.CR2.pp3 (11.1 KB)
I just switched tone mapping on and off —> auto match tone curve was not automatically enabled

With each change of any variable, curve, boolean or whatever else exists inside the GUI the pp3 is saved immediately.

I try to explain your problem with my words: if you clicked that huge button labeled “Auto-matched Tone Curve” and did not go back one step in the history window but simply applied further changes, well then you have data in your pp3 which you might not like.

I think this answers my basic question. This looks like it may be new material in RawPedia because I thought I was pretty well read on processing profiles. The first sentence says, “Processing profiles evolve from one version of RawTherapee to the next. We strive to ensure backward compatibility (e.g. a profile created in 5.3 and opened in 5.4 should look the same), but this is not always possible.”. I take this to mean “go ahead and try it but you can’t count on it to produce the same result.”. With respect to workflow, my interpretation is that if you want to preserve the state of the editing that has been done in a manner that either editing could be resumed from that point or additional exports made of the same result then you had better retain both the relevant processing profile and the version of RT used to create it. Based on this experiment & discussion I’m also inclined to think that one should be sure to make (i.e., save) any processing profile that is to be preserved in a way that RT will not confuse it for the ones that are automatically opened & rewritten without requesting user permission.

That leads to the question, “might it be better to export the resulting image to a file using a standard format that could then be reopened in either RT or any other image processing software?”. I think this would eliminate future ability to perform certain raw processing operations such as demosaicing. I’m not experienced enough to have an opinion about this but would be grateful to learn form those who are.

Of course the answer could also be to take both of the above actions.

If you tick this checkbox, your pp3 will be saved in addition besides your output file
image