Pipeline sequence in RT


(Andrew) #1

I’d like to better understand the order in which RT applies the various tools please, from start to finish.
Perhaps someone has some documentation, or can point me at something?

I could only find this by @jdc from 2 years ago -

…the process of RT:
a) raw (with Retinex, demosaic, WB, etc.)
b) rgb with tiles (RGB black and white, exposure, etc.)
c) Lab: without tiles (CBDL, Lab adjustements, etc.) for all processes except Wavelet (tiles and full image)

Thanks.


(Roel) #2

It may not help an awful lot, but most of the processing is done here: https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/improccoordinator.cc

It is… a little messy :stuck_out_tongue:


(Desmis) #3

@RawConvert
Here a simplified process…
Indeed, depending on the type of sensor, it may be different.
In addition there are 4 pipelines : dcrop.cc - improccoordinator.cc - simpleprocess.cc - rtthumbnail.cc

I hope I did not make too many omissions or mistakes

Preprocess
1 - darkframe
2 - Flatfield
3 - Badpixels
4 - Hotpixels
5 - Raw black point
6 - Lensprof - distorsion…
7- Green equilibration
8 - CFA linedenoise
9 - CA autocorrect - Chromatic aberration Correction
10 - Raw white point
11 - Raw histogram
12 - Prepare Auto Exposure

Demosaic
Retinex
HL recovery
White balance
Crops
Convert colorspace
Denoise - Noise reduction
Dehaze
Tonemap Fattal - Dynamic range compression
Automatchtonecurve
Tone response curve

Process RGB

  • channel mixer
  • tonecurve
  • highlight
  • shadow
  • rgb curves
  • HSV
  • Colortoning
  • Film simulation
  • Black and white
  • Labgrid colortoning correction (lab)
  • Shadows Highlight (lab)
  • Local contrast (lab)

Process LAB

  • locallab : blur and noise, denoise, vibrance, cbdl, soflight, local contrast, sharp, retinex, exposure, color and light, avoid color shift
  • Lab adjustements
  • Vibrance
  • Labgrid regions colortoning
  • Vignetting - Graduated filter
  • Tone mapping
  • Impulse denoise
  • defringe
  • edges microcontrast
  • sharpening
  • CBDL
  • Wavelet
  • Soflight
  • Ciecam
  • Post resize (resize, post sharpening…)

Final Lab -> rgb conversion


(Andrew) #4

Thank you @jdc for this. I’ve cross-checked against RT and have a few queries please…

I can’t see the following RT functions in your sequence -
Dynamic Range Compression (unless “Tonemap Fattal” ?)
Graduated Filter
Channel Mixer
Distortion Correction
Chromatic Aberration Correction
Vignetting Correction

a) is RT’s “Shadows/Highlights” covered by processRGB “highlight” and “shadow” ?
b) What is processRGB “Labgrid” and processLAB “Labgrid regions” ?
c) is RT’s “Sharpening” covered by processLAB “sharpening”
d) is RT’s “Noise Reduction” covered by section2 “Denoise”
e) is RT’s “RGB Curves” covered by processRGB “rgb”
f) does processLAB “Post resize” cover the whole of RT “Resize”
g) does processRGB “Film simulation” refer to the “Film-like” options in tone curves 1 & 2 in “Exposure” ?

Sorry to be picky but I think it’s useful to have this info available. Part of the reason I ask is that I’m wondering if you should make the changes in the same order as the pipeline. For example, whilst I don’t use Retinex much, it would never have occurred to me to do white balance after Retinex, and yet it might make a difference for all I know.


#5

While I don’t use Retinex in RT, I know it affects the colour balance somewhat, so for me it makes sense for you to adjust white balance afterwards. I think this applies to other tools as well: when you apply them, more often than not, there is a side effect that requires compensation.


(Desmis) #6

@RawConvert
Yes I had forgoten some rubrics, I just add and correct them

a) It is more complex than that…yes, but Shadows and highlihts are in many process
b) “Labgrid” and “Labgrid regions” are 2 mode of “Colortoning”
c) yes
d) yes, but only for wavelet / fftw / median
e) yes
f) yes
g) no (yes in part), But Film simulation is a rubric - you find just after HSV equalizer

But your question, under an implicit appearance is complex, what is in what, at what place put an action in a process ?
The answer is not simple and can not be summarized as “it’s better before” or “it’s better after” or “it’s better if we can choose the place”

If choose as example “Denoise” (generic term). The noise is a complex thing, some years ago, Emil Martinec (with my modest contribution) as work on noise. It was the time of DCRAW and Perfectraw

We differentiate, noise depending on whether it comes from the material or photons.
But in resumed, “Materiel noise” was at the beginning and “Photons noise” at the end.
Since we have work and migrate “Photons noise” at the beginning

Material noise : CFA lindenoise and a part treated by wavelet and bilateral filter

It is easy to say “Yes we can put elsewhere…” but “noise” is complex.
I think the best threatement is a distributed treatment

  1. CFA in first in RGB mode (banding)
  2. a minimum treatment in beginning , to avoid loss of detail, by using wawelet - fftw - median, and “impulse denoise” (bilateral filter)
  3. treatment by RT, which will probably increase the noise
  4. use of “Locallab denoise”, and or “wavelet levels”, to finished the treatment and adjust to what want the user. Leave noise in some places, accentuate the action in a general way

Other example with “Retinex”
Retinex is controversial tool, after being praised, he is now forgotten (a little), dehaze would be superior !

  1. retinex in RGB mode is incomplete, I propose an enhancement here issue #5099, by addind “Chroma” and “dehaze”, adding the qualities of the two algorithms, including “Retinex” which is able to separate front and background
  2. you have also "Retinex in wavelet (branch wavelet new)
  3. and also “Retinex” in Locallab" who allows to treat background and woks “after” at the end…

Retinex is a very complex tool, which I simplified the use in “locallab”

jacques


#7

Are you saying that Retinex needs to be updated, the underlying algorithms are older than dehaze’s, or both?


(Desmis) #8

@afre
I say, I have implemented a change in “locallab”, and for “dev” in issue#5099

This “new” algo, has globally the same code, but

  1. It take into account chroma (the actual version of Retinex desaturated images)
  2. it take into account “dehaze” and mixed the 2 algo

jacques


(Andrew) #9

Thanks again @jdc for all the info. I will look over this more when I have a bit more time.
I see tone curves are quite early on, especially the RGB one. (Ok, the ciecam one is right at the end). As tone curve is one of the most important adjustments, is there an argument for having them at the end, so you can make adjustments at the end of your RT editing session in the full knowledge of what you’re dealing with?


#10

I use the curves in Ciecam for at-the-end adjustments.


(Desmis) #11

@RawConvert @HIRAM

Yes, the"best" choice is CIECAM with all its (numerous) possibilities
Another way is to use “Wavelet” and Final Touchup.

Jacques


(Glenn Butcher) #12

I’d say so, in light of the recent discourse on scene-referred image data and processing it. Since I can arbitrarily order image operations in my software, I’ve been playing with keeping all tone modification at the end, even after output resize/sharpening, and I can easily say it doesn’t degrade the result. Need to look more closely to determine if there’s a specific benefit, but I’d posit that it retains color since you’re not arbitrarily shifting the RGB channels independent of one another until after all the other (chromatic?) work is done.