RGB calculation: Color picker and display (wrong), exposure module (solved)

darky · January 6, 2023, 8:34pm

Thank you Mica,

to my humble opinion the issue has nothing to do with DT working space and color profiles. It is just how the pixel calculation is done. May be you can try my posted PNG file. I doubt, that there is any special setting in DT to do the calculation and display correctly.

Here is a screenshot of my Smath calculation result:

In DT you can play with all profiles (input, display, working) and even with all set to sRGB (web safe) the color picker and display is incorrect.

priort · January 6, 2023, 8:56pm

Well for sure you can see the different pipeline processing or something going on for display/preview of this image… If I select 100% or anything greater directly from the dropdown I get yellow or green display… whereas anything under 100% zoom or any scrolled zoom will display as more the red…

Note main preview and zoom/pan preview are not the same.

200%

Arbitrary 172…

On my screen 100 and 200 are more green…this doesn’t show in chrome as I am looking at it here now…

darky · January 6, 2023, 9:15pm

Yes, thank you Todd.
For high zoom > 100% of course the display turns better, because you see the single pixel better.
Open the checkerboard file with Gimp and it is displayed correct for ALL zoom values.

This should always (like in Gimp) be the resulting
and correct effective (perceived) color:
mltc_rgb-188-188-128
This is RGB=188,188,128 and not the red with G=128
from the wrong DT color picker and display.

priort · January 6, 2023, 9:20pm

Y it was just interesting to see the different processing in DT 100 200 400 or any setting that I could go to directly from the dropdown looked like what you are showing… if I scrolled one click then I got the more red version… so 105 195 250 850…any percentage so DT must use a different scaling or processing for any zoom not in that list??

priort · January 7, 2023, 3:49am

Affinity photo seems to get fooled as well then

And other tools…

How did you calculate the number you found??

Seems like the color is correct and as you zoom out and the averaging increased the display gets closer to the color??

darky · January 7, 2023, 8:57pm

Dear Todd, dear Mica, dear DT experts,

the wrong sRGB color calculation is not a new thing and I really do not understand
why a sophisticated tool like DT does address highly complex modules
but does NOT address the basics of RGB calculation for display and color picker.

A couple of people raised the topic multiple times as a general issue
of most software tools for image browsing, (pre)viewing, editing, scaling …
Since 2007 (!!!) Eric Brasseur about “Gamma-Scaling-Error” on his website
www.ericbrasseur.org/gamma.html with the famous test image of Dalai Lama.

The math and solution is well defined in sRGB standard IEC 61966-2-1 since 1999 (!!!).
The standard has to be purchased, but all relevant formulas are published.
For example here: en.wikipedia.org/wiki/SRGB.
In the chapter “Transformation” you find the gamma-companded and gamma-compressed formulas. They are defined stepwise, a linear area and an exponential area.
On my website www.mtlc.eu/dam/srgb.html you can find a couple of graphics of this function.

The principle with sRGB is:
For any pixel calculations you have to convert the nonlinear coded sRGB values
back to linear values. Then you can process and make any manipulations.
For example the average calculation of color picker or resizing images.
After this you have to convert this result back to nonlinear coded sRGB.

I assume, that this calculation is done correctly in all DT modules with state “lin RGB” processing.
But I will check step by step, if this is really always the case. Especially the “last” module with scaling to specific output pixel dimensions is most important.

Without correct scaling of images in any zoom not equal to 100 %, the displayed image
will be wrong and the result is subject of misinterpretation!
This does not affect homogenous color areas, BUT it does affect any small details
(color and brighness variation between pixels).

How and to whom shall I address this DT issue? Thank you for any advice. May be a direct contact to the SW authors? I am hesitating to explain this simple basic topic in Github in detail. I can understand Aurelien, who complains about GUI extensions in DT. Better concentrate on the “hidden” basics. sRGB standard is now 24 years old and apparently still ignored quite often.

paperdigits · January 7, 2023, 9:27pm

The darktable github is the place.

darky · January 10, 2023, 10:31pm

Thank you Mica. Done!

Here is a good color picker: onlinejpgtools.com/jpg-color-picker
who does the color calculation exactly how it should be.
It can not only pick from jpg, but at least also from png files

You can start with from 1 px to more pixel eyedropper diameter.

Following is a screenshot with my above checkerboard and the correct effective color with 15 px radius.The checkerboard is not displayed correctly (scaling issue of all browser), but the measurement result is ok.

But you need to upload a file. That is why I would really like to have an offline color picker tool, with all these features! Or even better implemented in DT or/and Ansel!

priort · January 11, 2023, 3:49am

@dtorop Is working on updating the colorpicker code from what I can see… It would be interesting to see what he would have to say …

Its here if you are interested or want to provide feedback…

github.com/darktable-org/darktable

colorpicker cleanups/improvements

darktable-org:master ← dtorop:colorpicker-openmp-update

opened 06:23AM - 28 Dec 22 UTC

dtorop

+431 -735

Cleanups to colorpicking code. Removes > 300 LOC. There should be no user-visibl…e differences in most cases (excepting being a bit faster). Big changes: - Unify the iop, global, and live sample colorpicking code. Much of this code did the same thing, but previously it had multiple executions paths. - Add a flag to allow denoising to be turned on per picker. Denoising is helpful for filmic auto-tuning, but in other cases we actually _want_ to see the real minimum/maximum pixel values, noise and all. Denoising also is slow (5x slowdown, plus more memory usage), albeit for what is generally a very fast operation. And the current denoising implementation will denoise the entire image on the CPU path, even if the sample is only a point sample. In this PR, denoising is only enabled for filmic. - Add a flag which iop's must set to turn on sampling of their output. In the case of many iops, we only need to sample the input. In these cases, turning off output sampling is a 2x speedup. - Unify the sequential and parallel paths for colorpicker sampling. As before, parallelization is only used for sufficiently large sample sizes. But now this is done via an OpenMP if() clause, which eliminates a lot of very similar code. - Explicitly handle sampling of monochrome images without generation a warning. - Remove colorspace conversions before colorpicking in per-iop pickers. These were nop's. And as the colorpicker code is colorspace-aware, it should be the one to handle this if necessary, anyhow. Some more fine grained changes: - Use "worker" style functions (as in histogram) to allow for more succinct code. - Use custom OpenMP reductions (thanks to OpenMP > 4.0) in the parallel code, rather than handcoding effectively the same work. - When accumulating pixel values, only do the division to calculate their mean at the end. - Initialize the min/max/mean color picker values before sampling in all cases. (Previously, these were only initialized in the parallel case.) Use initial min/max values of `FLT_MAX` and `-FLT_MAX` rather than `INFINITY` and `-INFINITY`, as the former seem more in the spirit of IEEE floating point. - Use MIN() and MAX() macros instead of fminf() and fmaxf(). This should produce the same results, and allows the min/max code to use assembly opcodes vs. a call to libc. - Avoid possible division by zero for very small 1-channel X-Trans samples. - Print diagnostic if the requested colorspace conversion isn't supported by the code. - Use the `for_four_channels` and `for_each_channel` macros when possible. These give SIMD speedups and allow for removing uses of `omp declare simd`. Here's some timing data using this PR to sample from a full preview (approx. 1,150,000 pixels). All times are in seconds. For OpenMP, times are elapsed/CPU, for non-OpenMP time is CPU. ``` non-OpenMP OpenMP RGB -> RGB 0.003 0.001/0.007 Lab -> Lab 0.003 0.001/0.007 Lab -> LCh 0.027 0.006/0.053 RGB -> HSL 0.013 0.004/0.025 RGB -> JzCzhz 0.127 0.025/0.245 X-Trans 0.002 0.001/0.005 Bayer 0.002 0.001/0.004 ``` For single pixel samples, in all cases the results are 0.000 sec. elapsed/CPU, whether using OpenMP or not. Tests on a i7-10750H. For sampling with OpenMP, turning on denoising takes 0.005 secs. vs. 0.001 sec. without denoising. For comparison, here is timings with OpenMP before the current set of changes (using commit 105e72b89ec6264d5b0be27a3346347f11f6d7fe): ``` box picker point picker RGB -> RGB 0.007/0.051 0.005/0.013 Lab -> Lab 0.008/0.050 0.005/0.007 Lab -> LCh 0.015/0.109 0.005/0.009 RGB -> HSL 0.011/0.070 0.005/0.010 RGB -> JzCzhz 0.031/0.277 0.007/0.040 X-Trans 0.015/0.170 Bayer 0.015/0.140 ``` This isn't a totally fair comparison, as the timings of the current code are made without denoising. The 0.005 sec. overhead of denoising results in most of the speedup. On the other hand, the new code is no slower than the old code. And the old code would often unnecessarily run the picker on the output, hence doubling the sampling time.