Some news about RawTherapee’s Local Adjustments and recent Rawpedia updates

Some news about RawTherapee’s Local Adjustments and recent Rawpedia updates

  1. Local Adjustments (LA)

I have implemented an “automatic” full-image mode in the GUI so that you can use LA on the whole image. All of the LA tools can be used this way taking advantage of LA’s deltaE (Scope) and masks. To use it is very simple; in the menu “Settings” - Spot Method, choose “Full image”.

The LA Denoise module has been enriched with new features, such as: a hue equalizer, “denoise based on a luminance mask”, edge detection for DCT and “Recovery based on luminance mask” which allows you to differentiate between uniform areas and areas with detail when denoising. A similar functionality exists for “GuidedFilter”.

All modules are equipped with masks now (well, almost all of them) to allow the user to improve the deltaE selection. e.g. “Recovery based on luminance mask”

https://rawpedia.rawtherapee.com/Local_controls#Improved_result_with_.27Recovery_based_on_luminance_mask.27

  1. Rawpedia

There have also been a number of updates to Rawpedia.

You can see a new worked example of the simultaneous use of “Log encoding”, “Ciecam16”, “ Excluding spot” and “Recovery based on luminance mask”. Worth noting is a small improvement to the “Highlight reconstruction” - “Color propagation” performance thanks to the work Alberto has done on ART
https://rawpedia.rawtherapee.com/Local_controls#Log_Encoding_and_Highlight_Recovery

Some general information has also been added to help better understand the colorimetry aspects of RT, Ciecam (and in particular the addition of Ciecam16), as well as some explanation of the various denoise options in the Detail tab, the Advanced tab (Wavelets) and Local Adjustments.

Colorimetry :
https://rawpedia.rawtherapee.com/Toolchain_Pipeline#Colorimetry

Denoise :
https://rawpedia.rawtherapee.com/Comparison_of_the_3_Rawtherapee_noise_reduction_tools

jacques

13 Likes

Those are welcome expansions. And, without having tested out the newly implemented parts yet, especially the full image mode!

And updates to Rawpedia are always a welcome change.

Thanks Jacques!

Jacques.

1 Like

Well done Jacques for putting so much work into it over the years and adapting it into a much more user friendly form. This is a massive step forwards for RawTherapee.

3 Likes

Hello everyone,

Well done Jacques for putting so much work into it over the years and adapting it into a much more user friendly form. This is a massive step forwards for RawTherapee.

I totally agree!
It is also a huge benefit to take adavantage of the RawPedia documentation which is really well-done and updated :slight_smile:

4 Likes

Thanks, the recent documentation efforts are very helpful! I like the addition of the full image “spot”. I previously worked around it with a preset that had an oversized spot predefined.

I’ve mostly used the log module to handle dynamic range issues and occasionally I’ve used the vibrance warm/cool to desaturate and warm up daylight coming into a tungsten interior. The local lab tools are slowly becoming more and more usable!

It seems to me that even with gamut clipping disabled most of the tools are unable to recover highlights? I’m glad you address some of this with the recent wiki entry.

I still wish the spots would use a more common ui paradigm. Select shape icon > draw shape rather than dropbox spot type > click add spot > move spot > resize spot at four nodes

1 Like

Hello @jdc

First off, as a disclaimer, I am a long time user of RawTherapee but I never had the need to really get the most out of its denoise tools. Therefore I consider myself just a beginner (RawTherapee is indeed amazingly powerful to say the least…).

At work, as a plant pathologist, I take macro pictures, on a daily basis, with a Nikon D850 - ISO 64 (hardly any noise in the end…), k 2650 (Elinchrom lamp) on a tripod.

This being explained, since you are a master on the color grading theory I am wondering whether you think it is possible to achieve the same (or better) results compared to commercial softwares (yes, it is a very generic question sorry about that…)?
More precisely, I have read about the excellent results produced by DXO PhotoLab 4 as regards their new noise redection algorithm named DeepPrime [1]. Since it looks “easy” to use it today I have downloded a trial version on Windows 10 to test it.
On their site there are some RAWs file which are available to download and I have tested:

To get to the point, I have tried their noise reduction tool and later on, with the same RAW image, I have tried the noise reducions tools with RawTherapee:
(RawTherapee_dev_5.8-2723-gea6bb8fcf_20210115)
RCD-VING4 + Impulse reduction + Noise reduction (colour space lab > Luminance control: curve etc). That’s it: NO other RawTherapee tools used such as sharpening etc.

Here is my poor attempt result with RawTherapee (but I am 100% aware I am not an expert…) where there are 2 TIFFs (16 bit) side-by-side:

Or

Here is the Tiff (16 bit - integer), see screenshot above, exported from PhotoLab 4, where I have only opened the Raw and applied the Noise reduction tool (no sharpening, color grading etc):

In case you are interested, here is the RAW (Nef) file, with the eagle image, used for this test (as explained you can currently download it, for free, from their site):

[1] https://www.youtube.com/watch?v=m3HJiAJAkPs&t=233s

The latest changes are not in the latest AppImage (Jan 15) yet, are they?

@betazoid

All this change are only in “Dev”, since commit 2ea7b02

Jacques

1 Like

Hi! I’m so looking forward to this (I’ve waited years for LAs in RT), but haven’t been able to test it out as I’m currently on macOS (and no nightly builds are available).

I have a question regarding the processing pipeline. I’ve tried to follow the development of LAs here on pixls.us and I think I remember reading that the local adjustments will be applied after the “global” adjustments in the processing pipeline. Is that correct? If so, will the below quoted “full image” option is address that? Will that give a way to apply some full image adjustments after the local ones?

Please take a look at @HIRAM’s builds https://keybase.pub/kd6kxr/

@Silvio_Grosso Your question regarding denoise is better suited for a different thread since it is not related to Local Adjustments. That being said, if the name DxO gives their tools make any sense, they may be using a genetic algorithm / deep-AI-learning-whatever algorithm, which is unlikely to find its way into RT.

1 Like

I’m a Mac user and I run the nightly in Linux via Parallels - it’s worth the Parallels license
fee believe me!

@jdc - as ever Jacques, your work is very much appreciated.

2 Likes

Nice work performed on this dev branch.
Any news about the eventual integration of local control in the main branch? Is it planned to released it in the next 5.9 release?
Thanks for your answer!
regards
olivier

The tool youre looking for is wavelets denoise. Check out the YouTube video by @Andy_Astbury1 on how to use it effectively. From memory I think it’s this one (but he does have a few others on noise reduction):

After seeing the results, you probably won’t feel the need for commercial software.

@cabernet_olivier
Yes “Local adjustments” is in main branch, but not yet in 5.9…I don’t know when exactly :slight_smile:

@Andy_Astbury1
Thank you very much for your evaluation :slight_smile:

@mikae1
“LA” is positioned in the middle of the pipeline, just before the global Lab settings.
https://rawpedia.rawtherapee.com/Toolchain_Pipeline#Processing_order

You can put as many spots as you want, but it seems more logical (not mandatory) if you want to use a “full image” spot to start with this one.

Another remark, these “full image” are not new, just now it’s automatic. It used before to take 6 manipulations, now it takes one.

General remark : I recommand to clean the cache to avoid bad behavior with this new GUI fonctionnality.

@Silvio_Grosso

The problem you mention is not simple and is more than a general problem of “denoise” as pointed out by @Thanatomanic

Nevertheless I evoke the problem of the noise, in one of the links that I gave with in particular the possibility of “marrying” processes; for example “Denoise (main)”, with “Denoise LA”… since we are in an LA thread

I’m going to propose an approach, I’m not saying it’s the best…nor the worst, the denoising and the appearance of the image is something subjective that depends on a lot of parameters (personal tastes, shooting conditions, viewing conditions, material used, phisiological abilities (eg I am 73 years old and my visual abilities are no longer very good. …) things that partly come out of a CAM (Color apperance model)

First of all, the image setting parameters (independent of Denoise) are different in the Dx0 and Rawtherapee image.

if we look at the eagle’s wings, the black parts and the white parts :

Black wings :

DxO L=2, RT neutral L=6, RT « auto-matched » L=11

White wings :

DxO L=94, RT neutral L=41, RT « auto matched » L=60

In the same way the image is more saturated, more flattering

I don’t own DxO, Capture One or Lightroom, and my computer is only used for development RT. So it’s hard for me to make comparisons. Nevertheless the principles of CAM (human vision) are there. For example we will see much less chrominance noise on an almost black background, than on a gray background. I’m not trying to say, it’s better or worse, but different and evaluators will have to take that into account. it is therefore necessary to try to evaluate the noise by « ignoring » these differences.

How did I do it? By remembering that it is one solution among others. For example I didn’t use “wavelet levels denoise”, nor “excluding spot”…and some of my choices are arbitrary

First step, I open the raw file, with default settings (with auto-matched tone-curve)

Second step : I used « Noise reduction (main) » with « small settings » with 2 goals :

  • remove large clumps of noise

  • reduce a little bit the noise on Eagle

Of course, these settings are quite arbitrary and can be changed.

Third Step : Using « Local adjustments »

Some principles used :

  1. avoid using tone curves and prefer the “tone equalizer” to change the general aspect of the image (contrast…)

  2. Step 3 – 1 - Denoise :

differentiate the action between the green background and the Eagle. For this we will use principles and methods :

a) Work in full image

b) position the center of the spot in the zone to be denoised the most (the green part)

c) choose a high scope value, because the range of the image goes from green (background) to red (Eagle), so big amplitude of dE, but not to maximum

d) use the hue equalizer, to strengthen the action in the greens and reduce it for Eagle (red)

e) create a mask where we will try to separate the eagle from the background (the background in black)

f) use « Recovery based on luminance mask » to accentuate denoise on the green background

g) use a curve « Luminance denoise by level » minimizing the action on the first level to preserve details and then acting mainly on the lower levels

h) use « Fine chroma » to adjust chroma noise

i) others settings (Luminance detail recovery, Chroma detail recovery, equalizer, edge detection…) according to your taste

j) note the importance of the bilateral filter

  1. Step 3 – 2 - adjust the micro-contrast and saturation of the eagle

a) Create a new RT-Spot around the « Eagle »

b) use « Vibrance warm cool » to increase Saturation

c) use « Local contrast and Wavelet » to increase « local contrast » by using « Contrast by levels ».

All these settings are quite arbitrary, you can for example change the shape of the curve “Luminance denoise by level”, or the DCT parameters, or Bilateral, or “Recovery”, or…
Eagle_copyright-Jean-Charles_Rivas.NEFjdc.pp3 (22.4 KB)

4 Likes

Thanks! Now I’ve looked at the processing order and this kind of brings me back to my first post on this forum. It was not very well articulated, but I still stand behind the principle.

I so wish that at least Film simulation would be placed after the local adjustments. As an example… The LUT usually increases contrast and thus darkens shadows. Trying to raise those darkened shadows — during dodging and burning via local adjustement — after applying LUT will reduce the editing headroom for local adjustments dramatically. It seems counter intuitive, but I’d love to hear the reasoning.

To me, the logical placement of the local adjustments in the processing order would be between step 12 and 13.

I want to add that I’m extremely thankful giving us these local features. I also want to add that I’m not a programmer, but I did work as a professional full time retoucher for close ten years and I this is the way those I came across worked. If possible, local adjustments first (for maximum editing local adjustments headroom) and global after.

Now, I need to install VirtualBox ASAP to try this thing out! :grinning:

Yes, as it’s in dev branch it will also be in 5.9

1 Like

I think this makes sense, but I’m not sure if there is a catch. If you would, please file an issue on our GitHub tracker so that we can discuss it there and make code suggestions / test cases.

As I mentioned before, you don’t need to. HIRAM makes macOS builds of the dev branch which are available here.

1 Like

Oh, I read 5.8-xxxx on those and figured they wouldn’t contain the the local adjustment features that will be a part of 5.9. Thanks!

1 Like

Hello @jdc

Thanks a lot indeed for all your suggestions. They are super useful (I have even printed them on paper…).
I am extremely grateful for all your work done on RawTherapee all these past years. The Local Adjustements are quite powerful :slight_smile:

I am going to study all your Local adjustment process thanks to the .pp3 file you have linked on this post, for this Raw image :slight_smile:

1 Like