Research and development - White-Balance: auto -temperature correlation - tests - finding the optimum settings

Software RawTherapee
61

Faced with all these indeterminacies, I made several choices to reduce the complexity

  • analyze 237 colors of the image (instead of all). Of these 237 colors, only the xy (xyY) values are kept, they cover the entire visible spectrum.
  • choose a patch among these colors which is the most representative. After many tests this patch is made up of the 20 to 70 most represented colors. Their impact is weighted to take into account their frequency.
  • To develop this patch you must start from a starting base. It’s either:
    “Camera” values (temperature and tint) by eliminating tint values too different from 1.
    “Automatic WB grey” values if “Camera” values are probably “out” (tint > 1.5, temperature < 3300K or temperature > 7700K) and if “Auto WB grey” values are not themselves “out”

From this patch and these starting points, we calculate the corresponding spectral data 20 to 70 among 406.
From these various data we vary “temp” (± 200K approximately,) and “tint” (green), to find the best correlation.

If the found correlation temperature of is in the range 4000K, 7000K (most common daylight), keep the result and run a single pass. If the values found are < 4000K or > 7000K, a 2nd pass is performed with a new value located between 4500K and 5300K. The algorithm is restarted, new starting values, new patch, new spectral data. The choice is made by optimizing the product deltaE and Correlation factor.

In the case of 2 passes, you can select either one with the “Remove 2 passes algorithm” checkbox. The number of passes, the size of the patch, the deltaE, the correlation factor, appear in the GUI.

Due to the increased number of spectral data (200 to 406), the noise reduction (median 3x3) applied to the image data, the number of passes, the processing time has increased significantly, passing (depending on the images and the machine) from 100ms to 500ms. So, one can hardly consider a “longer” treatment.

As I mentioned, this system works correctly in the majority of cases, but it fails (like all systems…) in a few cases. Aiming for 100% is an utopia.

I updated the documentation on Rawpedia (simplified), and I think that apart from a few changes - especially at the GUI level, this version is almost complete. Thank you to everyone who contributed to this significant improvement

https://rawpedia.rawtherapee.com/White_Balance#The_Temperature_Correlation_Algorithm

And the last version
https://github.com/Beep6581/RawTherapee/releases/tag/pre-dev-github-actions

Thank you again

Jacques

62

Hello @Jacques,

Just tested the new build (5.9-349-g25ce3c99f_win64_release).
It does work extremely well on the majority of my Nefs images.
Most of all, I have noticed it performs well with outdoors images (flowers, cars etc) with no artificial lights (lamps etc)

Thanks a lot indeed for your efforts!

63

@Silvio_Grosso

Thank you very much :slight_smile:
jacques

64

^This is important info for me and mirrors how the tool “feels”. In a sense it’s not automatic white balance but white balance refinement based on info from the camera or Auto WB grey.

My question is then if itcwb could work off custom values? Use colour picker or manually input values as a starting base. This would allow the user to guide the algorithm towards the parts of the image deemed important by the user. Perhaps this is antithetical to the whole project but from my testing and feel of the tool this would be useful.

Itcwb would then not be a separate white balance tool but one you can apply “on top” of the existing white balance settings. The wb dropdown would then have the normal modes.

  • Camera
  • Automatic
  • …
  • Custom

Itcwb would be an option on each mode. Perhaps just a checkbox as the defaults seem quite good. Personally I would use a preset with Camera + Itcwb. For images where I made a mistake (see above) I’d use Custom + Itcwb.

65

Of course, you can change the name and the place in the GUI, however some parts come out of an “auto” mode (as “auto wb grey”), and other parts of the mode “refinement”

I will (for “custom”) have a look, but it seems difficult…

Jacques

66

My technical understanding is extremely limited but perhaps such a “checkbox” workflow would eliminate having to set difficult thresholds for which starting base to use? That decision would be left to, and controllable by, the user.

67

I think it’s (very) difficult to implement, because you want both an auto process (with bias for example the “bias” allows you to change “temp”) and a manual process with an input value on demand. Complex at the GUI level (not my strong point), and at the RT process level. On the other hand quite “simple” (??) at the algorithm level

jacques

68

If you want to “easily” and significantly change the temperatures, you must check the box “Remove 2 passes algorithm”, otherwise the algorithm will no longer understand anything, and act on “awb temperature bias”. The system will recalculate “tint” (green), etc.

jacques

69

A quick mockup to show what I imagined. Naturally the Multipliers, temp, tint,blue/red would change after activating itcwb but I left that detail out of the screenshots.

Default workflow with white balance from camera exif

rt-itcwb-as-checkbox
rt-itcwb-as-checkbox817×530 35.9 KB

Itcwb option when using “Custom” white balance option through picker or manual tweak. The point would be that itcwb would then base itself around the set values.

rt-itcwb-as-custom-checkbox
rt-itcwb-as-custom-checkbox817×530 36.6 KB

The panel would look similar for “Auto whitebalance”. If you check “Auto temperature correlation” values wb will be recalculated. Unchecking it should then set the “Method” to Custom regardless of starting method to allow fully manual tweaks of the calculated itcwb values.

My understanding is that you have been trying to find the point when camera wb is obviously bad and then switch to auto wb grey for the base of itcwb. You have also explained why this is necessarliy arbitrary. By coupling it with White balance “Method” (dropdown menu) you avoid this arbitrary decision and leave it to the user to choose.

I have very likely misunderstood large parts of itcwb and RT so please take this with a large pinch of salt! Unless you want to there’s no need to explain, you’re doing lots of work and don’t let me distract you to much!

Edit: I realized the manual sliders should be greyed out with only the bias active. Changed the images accordingly.

70

I have just made a “commit” (in wb_research bfb7d19) which changes, for the “camera out” images, some adjustments in particular on the “greens”. This approach can only be empirical, because “camera” is “out”, “Auto wb grey” is “out” or doubtful…
I also corrected the bad (but which does not change the result) display of the correlation coefficient.

Regarding the taking into account of “Custom”, as I mentioned, the problem is not at the level of the algorithm, it’s even quite simple, but at the level of the GUI.

The proposal that I understand you are making is (at least for me) at the GUI level impossible to implement, because it supposes a complete overhaul of the GUI (which here for White balance is - at least for me - very complex).
I got around the obstacle a bit, from my point of view, in terms of use, it’s not good, but it works.

After a cycle of “Itcwb”, you must go to “custom”, change “temperature” and “tint”, then return to Itcwb, and check the box : “Use custom temperature & green”.
I don’t know with the GUI update “Custom” when Itcwb worked, because it’s the cat biting its tail. As the system is planned (“auto mode” which from my point of view is essential in at least 50% of cases), it is difficult or even impossible (at least for me, the GUI is Chinese, I am a scientist and not a computer scientist).
But it allows to see, according to the images, how the algorithm behaves…
From my point of view, it is not very usable. AWB temperature bias (even if it doesn’t do exactly the same thing), it works.

Jacques

71

Thanks for doing this! What it shows me is that it is indeed possible and useful to “guide” the ictw algorithm using the colour picker. I searched for images with white objects in shade and in direct sunlight and used the picker on the shaded respectively sunlit parts of the white. itcwb then tuned each version to produce an image with a seemingly improved whitebalance for each respective lighting condition.

The interface is (understandably) buggy and it’s hard to know what data is used for the itcwb. However what seems to work for me is the following.

  1. Set processing profile to neutral (overwrite)
  2. Set whitebalance method to “Temperature correlation”
  3. Check “Use custom temperature and green”
  4. Press w, or otherwise activate the wb colour picker and pick a neutral coloured part of the image
  5. Switch WB Method back to “Temperature correlation” and observe a small tune of wb.

You can then repeat by starting from point 4 again, running the algorithm on different parts of the image.

What is your evaluation of this guided mode of itcwb? Gui apart, I fully understand that it may not me feasible with the current devs to create a reasonable ui.

To me it really shows just how dependent on the camera wb the default implementation is. I don’t think calling it auto wb makes sense when it takes such a large cue from the camera meter. It’s more like a white balance refinement? Does this make sense to you?

72

@nosle

Still in “wb_research” 84d3f2d, I made 2 small changes: 1) change the 2nd temperature (arbitrary) when the reference is lower than 4000 and put it closer to the first; 2) replace in the GUI “Automatic” by “Automatic & Refinement”.
For the “Use custom temperature & green” addition, unless you want to keep it (I don’t think you should keep it, because it’s not very functional and intuitive), I’ll neutralize (and not remove) it on the next commit. I don’t know how to make this kind of changes, which would suppose to define specifications - what GUI functionalities do we want? (appearance as you show which is one thing and possibility of iterations like bias, etc.,… but not for me !!)

Jacques

73

I just push a change in the PR. The same as in “wb_research”, but with “Use custom temperature & green” neutralize.

https://github.com/Beep6581/RawTherapee/releases/tag/pre-dev-github-actions

jacques

74

I just added the equivalent of “AWB temperature bias” for the “tint” by adding a “Green refinement” slider. (wb_research 2e5c101)
Allows you to change the “tint” (green) which will serve as a reference when starting the algorithm. It has substantially the same role for greens as “AWB temperature bias” for temperature.
The whole algorithm is recalculated. Depending on the case, in order to guide the algorithm, the action on “Remove 2 pass algorithm” may be necessary.

I don’t see why, I didn’t put it earlier, it’s simple (at the GUI level), and acts on the starting “green”.
The “green” is more difficult to find than the blue/red balance, because it depends on other things:

  • the quality of the illuminant
  • of the auto WB method used by the manufacturer
  • etc.

I think it has much the same effect as “Use custom”, only considerably simpler.
Moreover, there is only 1 (simple) line of code.

Executable versions are in production
https://github.com/Beep6581/RawTherapee/releases/tag/pre-dev-github-actions

I just did (10/05/2023) some cleanup in wb_research and added a forgotten variable (itcwb_green) in part of Procparams.cc. This should improve the behavior of the new “Green refinement” slider (at least I hope)

jacques

1 Like
75

Hello

A bug was found concerning the deltaE problem… It only worked if Verbose=true… (display in the console).
So corrected (wb_research fc876dd), it should work much better especially in case of “2 passes algorithm”. .

Executable versions are in production

Jacques

76

Hello Jacques,

Just tested the new build on Windows 10 (RawTherapee_6643merge_5.9-360-g1c3fa91e4_win64_release)
With most of of my images (Nikon D850 - NEFs), about mushrooms, the results are really odd (to say the least) :slight_smile:
From your previous suggestions, I suppose this might occur due to the nature of algorithm.
With the WB custom setting for the “camera” of RawTherapee everything occurs fine (the white balance is pretty much perfect)

I shot all of them with a manual white balance: around 2850 kelvin (Nikon D700 and Nikon D850). This specific setting in kelvin was due to a previous measurement with a Kenko color meter (KCM-3100).
The lights were professional lamps: no other natural light came from outside (no windows in this room). Therefore, the light (Kelvin) was always the same amount every single day :slight_smile:

Here is a short video with all my steps:

My post is just out of curiosity.
No need to change anything on your side.
I understand I am supposed to always prefer the automatic setting of my Nikon cameras for the white balance (no the Kelvin manual ones)

This is the link (Microsoft - OneDrive store) of the NEF image of the video just in case you are curious:
MUSHROOM

77

This latest build now handles my accidental wb building shot very well. Will do further testing.

78

@Silvio_Grosso
As I said, it’s a utopia to believe that everything can work in all cases. The system is not intelligent, but responds to a logic.
I changed some (arbitrary) threshold settings to keep the “Camera” settings in some cases. This should work better in your case, without hurting the others too much (Note, it’s not the Itcwb algorithm that is at fault, but upstream choices)

@nosle
I reactivated “Custom” by modifying within the algorithm, the possible fluctuations of “temperature” and “green” when we are in the mode “Use custom temperature and green”. I think now, it works and I think acceptable (??) how to use it.

All these changes are in the “wb_research” (d70a6c9) branch, I will check the operation before porting it to the PR (with executables)

Thanks for testing.

Jacques

79

I made some additional changes (which I should have done a long time ago!), in particular to optimize the search range for “greens”. This has 2 advantages: 1) we save processing time, 2) we avoid outliers.

I also cleaned up the code, and made other minor optimizations.
I also updated Rawpedia
https://rawpedia.rawtherapee.com/White_Balance#The_Temperature_Correlation_Algorithm

I just updated the PR, the executables are available.
https://github.com/Beep6581/RawTherapee/releases/tag/pre-dev-github-actions

But of course nothing is perfect

Jacques

80

Hello Jacques,

Just tested on Windows 11 - RawTherapee_6643merge_5.9-366-gf178f39fd_win64_release

Everything works fine.
I have tested all my NEFs images - manual white balance (mushrooms etc)
Thanks a lot!