Any interest in a "film negative" feature in RT ?

Here is a negative I just processed with RawTherapee using your article, except for flat field, as I digitized the negative a few months ago.
Original film: 6x6 cm color negative, don’t know the type. A shot my father took about 1960.
Digitized with Nikon D800 and 60mm f2.8G micro Nikkor.
Your article has definitely improved my processing color negs with RT. Thanks, and many thanks to rom9 for creating such a great tool.

HS3_5424 Long beach 1920|690x654

4 Likes

Thank you, I’m glad it helped! Now let’s hope @rom9 succeeds in keeping the process described in the article in the new version.

1 Like

Is there any advantage to picking neutral spots in the film negative tool for B&W negs?

Hi all :slight_smile:

Yes, i’ve found a solution that should keep perfect backwards compatibility and let the user choose between old method (invert before input profile) and new method (invert after input profile).
Unfortunately i’m progressing very slowly since it’s been a crazy time at work lately. Next week i’ll be on vacation so i should have some time at last :wink:

Frankly, i don’t know if i would even use the filmneg tool at all with B&W negs. The tool doesn’t do anything specific for B&W, so it could inadvertently “amplify” some slight color tint coming from the backlight.
I would simply enable the B&W tool (to neutralize any color tint) , then use Tone Curve 1 for inverted level adjustment (straight segment from max to min) and Tone Curve 2 for tonal adjustments (s-curve) … and that’s it :wink:
If you prefer to use the filmneg tool, the best thing to do should be to set both red and blue ratios to exactly 1.0, without picking any neutral spot… at least in theory :thinking:

BTW: very nice picture! The quality that you can get out of a ~60 years old negative is outstanding!

2 Likes

I just tried your suggestion of using the filmneg tool, setting red and blue ratios to 1.0, and also the reference exponent to 1.0. As well, I set the tone curves as suggested by Ilya, with curve 1 to define the right and left of the histogram and curve 2 to set a curve. It seemed to work very well and it seemed easier to avoid blowing the highlights than before, while keeping a pleasing tonal range.
6x6 B&W negative, shot in Puerto Vallarta 1953
Digitized using Nikon D850 with Nikkor 60mm f2.8G micro lens!

4 Likes

That’s great to hear! Looking forward to test it!

1 Like

Hello @rom9

Thanks a lot for reviving your work on this feature.
It would be super cool to have it ready for the next stable version of RawTherapee.
Just hinting, take your time and do enjoy your well-deserved summer holidays :slight_smile:

2 Likes

Here’s a colour negative test I did on a 6x6 cm Kodak (don’t know the film type) transparency. I processed the shot on the right back in January 2020 using the film negative tool of RawTherapee, but not really understanding it well. The shot on the left was from the tool on a recent developer version of RT, and after I had a better understanding of the tool from the great tips in this thread. The colours are much richer now, and I didn’t apply any saturation. The shot was taken by my dad in 1970 on Mt Robson as the Ralph Forster Hut was being built. Ralph is on the left.

4 Likes

Correction: The original Kodak film was a negative, not a transparency.

Hi! Sorry for the delay, can you try the filmneg branch again?
When you open a .pp3 file saved with the previous dev version, you should now get an almost identical result as before: the negative inversion will be performed in camera colorspace, but still after demosaic, so you get the best of both worlds: old behavior and new speedup :wink:

WARNING: please make a backup copy of the original pp3 files, otherwise you won’t be able to use them again with the previous dev version after opening them in the new version.

Let me know what you think :slight_smile:

Oops, sorry, the forum hasn’t notified me via email of your message! Sure, that’s super great news, going to try the new version and report here!

1 Like

Hi @rom9! I will post more detailed feedback as I gain more experience with the new version in coming days, but I just wanted to give preliminary feedback that this reworked converter ROCKS! I’m totally satisfied, I’m able to get even better colors than before! Thank you so much for bringing this!

1 Like

So, I’ve tried the new feature and here’s more detailed feedback.

Part I. Overall feeling.

I’ve tried it on several modern emulsions (Portra 160/400, Fuji Pro 400H, Profoto 100, Agfa Vista 200 (which is reportedly Fuji Superia 200) and compared with the results I was getting on previous development branch version (from April-May). Despite the behavior has changed with the addition of the new modes and changes of the WB slider, I was able to recreate and sometimes improve the look of the previous version. I liked that the new version feels more ‘stable’, I didn’t feel the need to nudge color coefficients after initial sampling while applying them to different frames on the roll. Previously, I had to do it for some frames which were overexposed, otherwise e.g. sky could turn magenta despite midtones being correct.
That being sad, the professional films (Portra/Pro400H) still feel harder to get right compared to the consumer ones. This was the case with the old version as well.

Part II. Questions.

  1. Can you please explain different modes of Inversion colorspace dropbox? What is the difference and when one is supposed to use them?
  2. Am I right that no matter what you choose in that dropbox, the new approach where main WB sliders control temperature of light shining through film still says the same?
  3. In your opinion, when is the best moment to sample film base, e.g. while being in “Working colorspace”? Is it after flat field application and setting main WB to lamp used? Or after all this and then after sampling neutral greys?

Part III. What would be good to improve:

  1. Blue balance slider in the negative panel which controls yellow/blue has orientation that’s reversed to the usual WB control in all apps (i.e. when you move the slider to the right, the expectation is that the image would become more yellow, while in RT it becomes more blue). Also, it’s inconsistent even to RT’s own WB tool. So would be good to mirror it from left to right. The green/magenta balance on the other hand is correct now.
  2. It would be good to put color dots to the left/right of the Green Balance and Blue balance controls, i.e. the same as it’s currently done in RT for regular WB. I.e. Blue balance slider would have blue dot to the left and yellow to the right, indicating what the image would turn to if you move into that direction.
  3. Tooltips are lacking for Inversion colorspace dropbox and Output level slider
  4. I’m really missing WB picker for the new Color balance sliders, as the ‘old’ color picker doesn’t work now (as the main WB now controls WB of the lamp shining through film). Can live without it, but it would be much easier with it.

Big thank you for implementing this! Looking forward to see it in the official release of RT 5.9!

2 Likes

Thanks for testing, i’m glad you like it :slight_smile:
I was already writing a small description of the new changes, i’ll post that first and then answer your questions below.

I just pushed some additional changes to make the GUI a bit more clear to the user (hopefully).
Here’s some reasoning behind this new version: in the previous (dev) version, there was the concept of a “film base” color. The user had to pick a spot of clear, unexposed film, and the RGB value of that spot in the negative image was saved in the processing profile.
During processing, the output multipliers (applied after exponentiation) were calculated so that the picked film base spot would yield a fixed gray level of 1/512th of MAX (2^-9).

Now, i’ve replaced all this with a more flexible concept of a “reference spot”: the user can pick any gray spot (not just the unexposed film), and the program will save two RGB triplets in the processng profile: the input values, sampled from the original negative image as before, and the output values, which define the corresponding output color that should be produced. The user can then adjust the output color via the additional sliders.
The processing code then imposes a very simple constraint:
“these input values, after exponentiation, should produce these output values”, and calculates channel multipliers accordingly.
This seems like a subtle difference, but it simplifies code a whole lot, especially when compensating multipliers for backwards compatibility; i don’t know why i haven’t thought of this before, i guess i’m an idiot… this would have saved me hours and hours of sleep deprivation :rofl:

As an added bonus, the user is now free to pick whatever reference color, even if not gray at all!
For now, i have set a constraint in the GUI so that the balance sliders are automatically reset to 1.0 after picking a spot (thus making that spot gray in the output), but you can always change the balance and make it a different color.
I think making it gray by default is a bit easier to use, because it resembles the usual spot WB tool… i’ve also labeled the button as such in my latest commit.

Finally, the reference spot is also used a “pivot point” for the exponentiation. For example, if you pick a mid-gray spot somewhere in the picture, and then adjust the exponents manually (red ratio / blue ratio), the reference spot will keep the same color, while everything else will change color.
This can be handy when it’s hard to find the right exponents.

Here’s the revised GUI:

image

Another important addition is the “Inversion colorspace” combobox. Currently there are 3 choices:

  • Camera space : is automatically selected when upgrading a processing profile from a previous version. When this is selected, all negative inversion happens before input colorspace conversion (but still after demosaic, so we still get the performance benefits).
  • Working space : means that the negative inversion happens after input colorspace conversion, and is performed in whatever colorspace the user has selected as his/her working space (this is what negadoctor does, i think).
  • FilmNeg (built-in) : is my hard-coded matrix that i described in the previous posts, fabricated based on the wavelengths of peak sensitivity of the film. Image is converted to this colorspace, the negative inversion is performed here and then the output is converted back to working space for further processing down the pipeline. This is the default choice for new processing profiles.

In the future, i could add a fourth “Custom” option, where the user can choose an arbitrary ICC profile. For now, to use an arbitrary profile, just select “Working space” and then set your profile as working profile.

Regarding when to use one or the other, i’m not sure. In theory, you should always use “FilmNeg” … but in some cases, like the examples you provided in the previous posts, the old version working on raw data was producing very good results; “Camera space” is there just to mimic the behaviour of the old version.
“Working space” allows you to choose other profiles if you want to experiment. It will use whatever working profile is currently selected. You can add custom working profiles via workingspaces.json

In short, yes. In practice, if you use the “Camera space” option, you can choose whatever WB values you want: since all the processing is done before the input color matrix, you will be able to compensate afterwards using the color balance sliders in the film neg tool.
Anyway, to avoid having to remember one more thing, i would say as a general rule: always use the backlight color temperature :wink:

Yes, exactly.

This is not required, you can sample the two neutral grays before or after

Ooops… sorry for that :rofl:

Agreed! I’ll steal the code from the WB tool :slight_smile:

Ooops, completely forgot, nice catch!

The newly-relabeled “Pick white balance spot” button should now do exactly the same as the normal WB picker :wink:

Thanks A LOT for the bug hunting, fixing this ASAP! I hope i’m not too late for inclusion in 5.9.

BTW, i apologize with everyone interested for the huge delay, unfortunately i spent most of the time trying to understand my own previous code. Back when i had only the film base color saved in the processing profile, making any compensation to the multipliers was a super-convoluted calculation, way beyond my reasoning ability.
Now, with the input and output values treated separately, everything is way easier.
As they say: “smart data structures, dumb algorithms” … so true :smiley:

Ps.: i noticed someone on YouTube complaining about film negative tool performance in RT5.7

… this to me is a confirmation that moving everything after the demosaic stage is a good move. This will be a happy user when the update lands :wink:

3 Likes

Thanks so much for your work on this. The filmneg tool is so useful.

1 Like

Rome was not built in a day :wink:

1 Like

On second thought, i don’t think this is correct; the usual WB control indicates the color temperature of the light source in a picture, while this slider simply indicates a multiplier for the blue channel. A more fair comparison would be to the channel mixer blue slider, which indeed works in the same way (the further right you go, the more blue you get).
Now with the color icons, it should appear more obvious.
I could see two alternative implementations to make things more user-friendly:

  • Having the 3 sliders indicate the absolute value of each channel (no ratios), with an additional checkbox labeled “Lock to gray”, whick links the sliders together to quickly set a gray color.
  • Maybe better, having 4 sliders: one indicating the “master” output level, and 3 sliders indicating a ratio for R, G and B channel. This would let the user also set the Red balance independently.

In any case, the purpose of these controls is to obtain an RGB triplet for the reference output value. Any idea is appreciated :slight_smile:

Here is the current, revised GUI with tooltip and color icons:

image

Appreciate your explanations! I’ve tried the WB picker and indeed, it’s got a bit easier! Have to find some time to test the FilmNegative inversion space, prior to your explanations I just ignored it not knowing what it stands for. Also I’ve only now realized that the film base control is gone with the last commit and it still works good, so that‘s cool, one variable less and you say it also simplifies code, so it’s great!

Regarding:

I think the UI should reflect feelings of a user, he doesn’t care if it’s some multiplier inside, for him the blue balance slider acts as “WB” but in reverse form, so muscle memory is confused (and the fact that the green/magenta is ‘normal’ also makes it a bit inconsistent).

Regarding making that as RGB controls, I think it would make matters harder, as now with the two sliders it’s quite intuitive where to push them to make the picture more ‘yellow’ or ‘bluish’, again, muscle memory from all the editing digital photos applies. I fear in RGB it’s more inconvenient, as you get one more variable to change while searching for good balance - in fact, the NLP plugin presents this and I dislike the experience, currently RT is more convenient in this regard with the two sliders. So I think you shouldn’t just replace it with 3 or 4 sliders, at least not without having the current way as an option. It’s important to be able to quickly change the WB, as just clicking on a ‘white’ with a picker doesn’t work in say indoors photo with incadescent lighting, you have to then bring the white balance lower to make the picture less blue. With more than 2 sliders it would be more juggling.

Rom9, where is that new Film Negative tool? I’m trying the latest appimages of both the regular RT and ART and it doesn’t look like that.

Color tab, below “Film Simulation” :wink: