The problem is exactly the same with “Green refinement”.
As I explained the GUI of “whitebalance” is complex, and coexisting an automatic process (with AWB bias, etc.) and a manual process (custom) is very complex at the GUI level (at least for me). I solved the problem by preventing when “Custom temperature and green” is selected, the use of “AWB temperature bias” and “Green refinement”.
If someone wants to tackle this GUI problem I see no problem…
To show the time spent by each major phase of the process, I have marked the algorithm with markers, if you compile, and activate “Verbose=true” you will be able to see for 7 parts of the algo, the respective times.
It allowed me to find where the time spent was important. I made an optimization by limiting the scope of the temperature scan (I attach a copy of the console)
“Fourth: from third recalculate spectral: 3283 nsec.”
With this change processing times are roughly divided by 2 or 3 (no OMP, no OPENCL completely useless in this case).
Don’t forget to “reset to neutral”, and then choose the pp3 profile you want, otherwise some new settings will not be applied.
Note: “live” tuning of this algorithm is fairly new (?) to Rawtherapee, as the algorithm is complex and requires a lot of back and forth (and excuse my bad english). Thank you again
Is it possible to recreate the sliders within the new tool and disable the ones above? That way if they behave differently, it’s not as confusing to the user.
Sorry, but I don’t understand this request. However, the addition of the “Use custom temperature and green” checkbox is very problematic at the GUI (whitebalance.cc) and “improccoordinator.cc” level. I don’t know how, and I’m not sure it’s doable.
So 4 options:
I delete the “Use custom…” checkbox and therefore the possibility
we leave it as it is now, and it’s up to the user (through back and forth to find a compromise), even if it’s not intuitive
I put the 2 sliders back, but the behavior will always be erratic (hard to accept?)
another developer is trying to see if it’s possible, but I wouldn’t make this request myself, because it seems difficult to me (the cat biting its tail)
I increased the temperature sampling, instead of 134 increments between 2000K and 15000K, there are now 175 (wb_research - b60262c)
Each temperature is associated with the white point (XYZ), for example
T=2605 , X=1.134667, Y= 1, Z= 0.290722
T=4914 , X=0.965682, Y= 1, Z= 0.806658
The calculations are more precise and may lead to small differences in the temperature found, compared to previous versions.
I guess i was basically asking to to separate the new white balance refinement tool from the regular white balance tool. That way if sliders behave differently, its not as big of a deal to the user.
I think I understand (??), but it’s an almost unsolvable GUI problem (at least for me). You cannot have a “with” and a “without” system at the same time.
It’s hard to explain, but the “position on the screen” won’t change the problem of the GUI and the calculation of temperatures, tint, and the action of AWB bias, and green refinment, and it doesn’t matter much to do with the algorithm.
Said in another way, whatever the choices, auto (the 2), camera, custom, at the end we obtain a temperature and a tint (the sliders are the same) which results in the calculation of the wb multipliers
the most important, from my point of view, is the increase in the temperature sampling which is now more “finer” especially around 2800K, 5000K, 6500K. In addition I extended it up to 15000K. This finer sampling may lead to small differences in the results found.
the “whitepoint” which is used in particular to size the patch and to determine “chroma patch” is now calculated from the illuminant matrix and Observer. On the one hand it is more precise, and on the other hand takes into account Observer 2° or 10°;
in order to allow the use of “Use custom temperature & tint” without imposing it - because it is not “ergonomic” I added in “Preferences” - “Color management”, a checkbox which allows this choice . This supposes restarting RT.
You have mentioned this and I have noticed that it is necessary. How come it’s not enough to reset whitebalance to camera and then re-set to itcwb?
I’m wondering because I think that it may be important to be able to reset the algorithm from within the module. Perhaps this is simply because it’s in development and will be solved when in release. It’s just important that future changes to code, after release, won’t require “neutral” to reset the wb algorithm.
I re-enabled (wb_research 9302100) an “auto” feature “update” that I had disabled while working on solutions. This should work “better”.
However some changes like the ones I made require erasing the pp3s (hence “neutral”), I now think that’s over. The algorithm now seems complete to me.
On the other hand the functioning of the system (it is not the algorithm strictly speaking) with “Use custom temperature & tint” is degraded and I think will remain so with very erratic operation.
Should we keep this option? From my point of view = no.
The gui and workflow with custom is certainly complicated and confusing. In the current implementation I don’t think it’s worth keeping.
The reason for wanting it is that I understood that itcwb isn’t really a white balance method on it’s own but in reality a refinement of camera whitebalance. Itcwb will follow what ever the camera/user set on the scene. Only to reset to use Alt_temp if the values are determined to be suspicious. It’s now working quite well and avoiding very wrong wb. It still makes warmer wb if the camera was set to that though.
I still feel itcwb, even in it’s current implementation, would work better as a tick box in the “Camera” method and removed from “Automatic”. It’s to connected to camera settings to be considered it’s own wb method. This is personal opinion and what I feel makes GUI sense. I fully understand that RT is lacking GUI coders and that I may have the wrong end of the stick! If it’s an issue of lacking GUI coding I think it should somehow be noted as a goal or wishlist to refine the GUI at some later stage.
Of course we can do semantics, but as the designer of Itcwb, and of a large part of the colorimetry of RT, I can say that Itcwb is not only a “camera follower”, but a real white balance. This is probably the most complex algorithm I have designed.
Admittedly, it’s also a question of point of view (and semantics) but Itcwb is at least as automatic as “Auto grey”, only considerably more complex…
If it was only an appendix to Camera, then why have more than 5000 lines of complex code, essentially matrix calculation on colorimetry? Why complicate your life by multiplying matrix of Observer, colors, illuminants (to my knowledge, other software uses often simple formulas without matrix). Of course nothing is perfect, the system fails sometimes (less than autogrey). Like I said no intelligence, only math…
The current GUI which takes into account many parameters (Itcwb is only a “small” piece, at the GUI level), is complex. Adding Observer (thanks to @Lawrence37 ) didn’t make things any easier.
I take note for “Use custom…”, Any other opinions on whether to keep it or not?
I just removed “Use custom temperature and tint” and that seems like a good decision. In a majority the use of “AWB temperature bias” and “Green refinement” makes it possible to do essentially the same thing.
As several people told me I reduced the “sampling” choices to 2 (as they were at startup) - “Low sampling” and “Force use of the entire CIE diagram” (default). As a reminder, this has nothing to do with the “working profile”.
I now think this version is finalized in terms of code, algorithm (of course we can change if necessary). It remains to optimize - if necessary - the labels and the tooltips. I am not an English speaker, so if someone can look and let me know these remarks.
After that I think we can ‘merge’ in ‘dev’, this PR as it is.
I just made a small modification, upstream of the algorithm. When you select “Low sampling” (probably to eliminate some colors, hence this choice), if some conditions are recognized (tint camera > 1.02, and others), then “Camera settings” are not used
But be careful, no miracle, there will always be unresolved cases
I did this morning (19/05/2023) an update both in wb_reseach (3ffbee8) and in the Pull-request (9cfa5ba).
The link for executable is the same as above
This version tries to better determine “upstream the algorithm” the temperatures and starting green, in the case “Low sampling & do not use Camera settings”
Reminder, this mode which does not use Camera data already exists “upstream the algorithm” and also inside the algorithm. But in this case we “force” the system (when green camera > 1).
Bonjour. Lien du test de ce jeudi 25 mai: https://drive.shadow.tech/s/mbESrMtfkBok7Sb
Un NEF et ppa. Les saisies-écran des BdB à tester (Low et Entire) et plusieurs BdB (APN,Lumière du jour,Ombre,Nuageux) pour comparer. Amicalement. Alain
Compared to my last interventions I make a point of the operation of the system to date:
modifications have been made upstream and just at the end of the algorithm (which is almost unchanged)
these last modifications (upstream and end) make it possible in almost all cases to offer 2 settings (temperature and tint) with indication in the GUI “Best or Worst” of the alternative solution.
3 samplings are offered:
Force uses of entire CIE diagram: This approach may seem unnatural, as in some cases it involves taking into account imaginary colors. Nevertheless, it is the one that is ‘normally’ integrated into the process (Raw process, demosaicing, etc.). The conversion to the “working profile” which can also handle imaginary colors comes later.
Medium sampling – near Pointer’s gamut: in this case I use the “Beta RGB” profile (B.Lindbloom origin) which seems closest to the visible colors in reflected color (Subtractive color mixing) - Rec2020 gives very close results .
Low sampling & Not use camera settings: this choice restricts sampling to sRGB (which roughly corresponds to the gamut of the Colorchecker24) and therefore eliminates high gamut colors. I have added to this choice another approach to the starting point (temperature, green) by calculating this starting point according to various parameters from a mix of “Auto WB grey” and “Camera”.
I replaced ACESP0 by JDCmax (new profile) for maximum sampling. The whole CIE diagram is not fully covered, less than ACESP0, but more than ACESP1 with different primaries. Imaginary colors may occur slightly in blues and greens with very high gamut(between 515nm and 530nm)
Primaries
Red 0.734702 0.2653
Green 0.0219 0.903
Blue 0.1206 0.0016
White point: D50
Just for information. If you compile either the wb_research (83e823f) version or the PR. You can see (it slows down the process) the display of the xy values found by the “sampling”. Thanks to Reffort for this work. These values are not sorted by number, so you will see values from 1 to 10 given…which of course are not retained. I would remove this feature (it can easily be re-enabled by compiling) so as not to slow down the system.
And if you activate “verbose=true” (options) and change the value of “Itcwb_deltaspec” (options), for example “Itcwb_deltaspec=0”, then you will see in the console the values retained and classified in ascending order. The smallest possible retained value (in number of sampled data) is 400*9 = 3600.
Compared to the display of data in the GUI, these can be different, eliminated for various reasons (for example being more restrictive for high luminance values, or x or y too small).
Thus you will see the influence of the sampling and the values chosen for the spectral data.
As a reminder, you now have another “working profile” at your disposal “JDCmax”. Large, less than ACESP0, but unlike Prophoto, imaginary colors can occur in extreme greens instead of blues, which from my point of view should be less of a consequence. I looked for a name, and many names have already been registered (alpha, beta, etc.)… This one must not already be taken
Version: 5.9-443-g932568e5f
Branch: wb_research
Commit: 932568e5f
Commit date: 2023-06-06
Compiler: cc 12.2.0
Processor: x86_64
System: Linux
Bit depth: 64 bits
My image below is now back to looking very warm. it sets a temp of 7002 for a sunny daylight photo because the camera was accidentally set to shade wb. For a while it was corrected to a reasonable daylight temp.
I also noticed another thing. If you set wb to temperature correlation using the file browser “batch edit” panel (after resetting to neutral) and export the image the wb doesn’t seem to be calculated using temperature correlation. It seems you have to open the image in the editor to trigger the algorithm. It won’t run on export even when set? It may be something I’ve done but so far my testing suggests this is the case.
