Color profile from Tiff file

[Guillaume]

Hi all,

I finally succeeded in having the correct calibration results . You will find below the files obtained without and with the saturation of the stereo microscope, side by side for comparison. The color difference values DeltaE are written on the images. I started from a maximum of 12.19 and decreased it to 3.67.

Uncorrected files

Calibrated files

Calibrated and normalized files

Here is my protocol

1) Pre-processing
(I took a picture of each patch under identical light conditions and assembled them together)

• RawTherapee
  → Input = output profile
  → Individual patches resize
• Gimp
  → Patches assembly
  → Global resize

2) Image linearization

• RawTherapee
  → Input = RTv4_medium_g2.2 (Adobe RGB)
  → Normalization (see post 60)
  → Exposure
  → Output = RTv4_medium_g1.0 (linear AdobeRGB)

3) Profile creation

• DcamProf
  → Illuminant = D65
  → Glare correction

4) Profile attribution

• RawTherapee
  → Input = DcamProf linear profile
  → Output = RTv4_medium_g2.2 (Adobe RGB)

5) Color difference measurement

• Darktable
  → Color calibration tool
  → Check the output DeltaE

More details about RawTherapee parameters
I used RTv4_medium_g2.2 because I knew that my files had a gamma of 2.2. You have to adapt the input gamma to your specific case. It’s important to note that sRGB primaries are not large enough to cover all the colors of the ColorChecker Macbeth chart, especially the cyan.

The clip out-of-gamut colors checkbox was unchecked and the white balance was deactivated. All the output files were obtained with absolute rendering and no black point compensation.

More details about the DcamProf script
I used the make-profile -i D65 option, the default being D50. It allowed me to have the same white balance in all my workflow. I removed the glare with -g, which enhanced the profiles. In addition, I forced linearity with make-icc -t linear. I didn’t use the -W white balance option, which was degrading the result.

DcamProf script

SET target=Zeiss_ICC5_sat0_231221_RT_g22_norm_lin
SET corners=20,20,1185,15,1185,795,21,792
SET argyll_dir=C:\Users\Guillaume\AppData\Roaming\DisplayCAL\dl\Argyll_V3.1.0
SET dcam_dir=D:\Photos\dcamprof-1.0.6-win64\dcamprof-1.0.6-win64

call :LOG > "%target%.log" 2>&1
exit /B

:LOG
@echo on
"%argyll_dir%\bin\scanin.exe" -v -p -dipn -F %corners% "%target%.tif" "%argyll_dir%\ref\ColorChecker.cht" "%dcam_dir%\data-examples\cc24_ref-new.cie"
"%dcam_dir%\dcamprof.exe" make-profile -i D65 -g "%dcam_dir%\data-examples\cc24-layout.json" "%target%.ti3" "%target%.json"
"%dcam_dir%\dcamprof.exe" make-icc -t linear -n "%target%_dcam_g10" "%target%.json" "%target%_dcam_g10.icc"
@echo off
del "%target%.ti3" "%target%.json"

I hope this thread can help other people in need of TIFF calibration ! Thank you to all for your fruitful discussions .

Regards,
Guillaume

[ggbutcher (Glenn Butcher)]

Already a big help, for some reason I thought the dcamprof default illuminant was D65. D50 it is, thanks for pointing that out…

[Entropy512 (Andy Dodd)]

@Guillaume

After using your Python script, the black image has RGB values arount 5/255.

I tweaked my script to analyze the data instead of plotting it:

There’s a mean of 1562, and if you zoom in the histogram has gaps every 16 values, so looks like it’s a 10-bit sensor.

Either the sensor has a black level offset or you’ve got some light leakage from somewhere. I’m used to capped-sensor images with just a black level offset having less spread than this…

It appears that some white balance or other color transformation has been applied, but the data was unfortunately quantized after that transformation.

The histogram is a bit easier to read if you divide all values by 16 first to get rid of the gaps:

Mean is about 95 for green and 98 for blue/red on a 0-4095 scale Also a WEIRD value for a black level offset, usually the black levels are aligned to a power of 2 (but not always…)

It’s too bad the CZI isn’t actually raw sensor data.

[Guillaume]

That’s very interesting !

@Entropy512

It appears that some white balance or other color transformation has been applied, but the data was unfortunately quantized after that transformation.

I can confirm that I did a white balance in the software, as well as a background suppression. The latter aims to remove the light gradient. These are the first steps I realize when I shoot photographs. The black photo was taken at the end of my series by turning off the light, capping, and decreasing the exposure time.

[Entropy512 (Andy Dodd)]

Ideally if you’re doing color profiling/management you want the sensor data to be as raw as possible, and handle white balance in postprocessing.

I’d be curious what happens with background suppression off and white balance disabled.

[Guillaume]

I don’t think I can disable the white balance. As for the background suppression, it considerably narrows the peaks on the histogram. As you said,

@Entropy512

It’s too bad the CZI isn’t actually raw sensor data.

Have a good day.

[Guillaume]

Hi all,

I just tried to add the Tone Response Curve (TRC) in the L*a*b* adjustment panel rather than in the exposure panel. It’s easier to obtain normalization with L*a*b*. You just need to calculate x from y=0 and x from y=100 and you can directly use these values in the TRC. You don’t need to correct the exposure value.

On the contrary, with the exposure panel, you need to adjust the curve with trials and errors. Moreover, after changing the curve in the exposure panel, you need to correct the exposure value.

By using L*a*b* adjustment panel, my calibration improves slightly, but the gain of ease is interesting. That’s a good tool to try !

[anon8280290]

@Guillaume

It’s important to note that sRGB primaries are not large enough to cover all the colors of the ColorChecker Macbeth chart, especially the cyan.

Only the cyan is out-of-gamut, and that only by a bit:

x,y = 0.21, 0.27

[Guillaume]

Hi @anon8280290,

Thank you for highlighting this point. You are absolutely right. Nevertheless, even one point being outside of the sRGB profile is enough to use a larger profile, such as AdobeRGB. Otherwise, calibration will be incorrect, or at least you will lose some available information.

@Guillaume

Here is my protocol
1) Pre-processing
2) Image linearization
3) Profile creation
4) Profile attribution
5) Color difference measurement

I am now able to process my samples pictures using steps 2 and 4 of my protocol. To automate it, I created a batch BAT program using rawtherapee-cli:

batch processing

::Script to process a batch of TIF images with rawtherapee-cli

::subdirectories:
::raw: place the original images, load the linearization profile
::       and adjust exposure in the graphical interface
::intermediate: load the calibration profile only for images with a different saturation
::       output linearized images
::final: output calibrated images
::web:   output sRGB images

::Images to process
SET photos_dir=C:\Users\Guillaume\Photos\
SET sat_val=1       & ::0 = no saturation; 1 = maximum saturation

::Define directories
SET RT_cli=C:\Program Files\RawTherapee\5.10\rawtherapee-cli.exe
SET RT_pp3_dir=C:\Users\Guillaume\AppData\Local\RawTherapee\profiles

::Select correct ICC profile
IF %sat_val%==0 (
    SET sat_pp3=%RT_pp3_dir%\Zeiss_ICC5_final_sat0.pp3
)
IF %sat_val%==1 (
    SET sat_pp3=%RT_pp3_dir%\Zeiss_ICC5_final_satmax.pp3
)

::Logging
call :LOG > "%photos_dir%\RT_processing.log" 2>&1
exit /B

::Active part
:LOG
@echo on
::Normalize and linearize images with existing pp3 (-S option)
mkdir "%photos_dir%\intermediate"
"%RT_cli%" -o "%photos_dir%\intermediate" -S -b16 -tz -Y -c "%photos_dir%\raw\*.tif"
::Apply ICC profile unless it already exist (-s option)
mkdir "%photos_dir%\final"
"%RT_cli%" -o "%photos_dir%\final" -p "%sat_pp3%" -s -b16 -tz -Y -c "%photos_dir%\intermediate\*.tif"
::Export for web
mkdir "%photos_dir%\web"
"%RT_cli%" -o "%photos_dir%\web"  -p "%RT_pp3_dir%\Web.pp3" -s -b8 -n -Y -c "%photos_dir%\final\*.tif"
@echo off

Here are some powders taken under the stereo microscope and calibrated.

[jdc (Desmis)]

@Guillaume

Several months’ work and your perseverance have finally produced a very good result.

These results, with a “semi-automatic” process, will enable constructive exchanges between scientists.

Jacques

[anon8280290]

@Guillaume

Hi @anon8280290,

Thank you for highlighting this point. You are absolutely right. Nevertheless, even one point being outside of the sRGB profile is enough to use a larger profile, such as AdobeRGB. Otherwise, calibration will be incorrect, or at least you will lose some available information.

Yes, I see that your work is much more scientific than mine.

As to “calibration” it is interesting that so many profile creation apps are based on the 24-patch card. I wonder if all such profiles are a little off in cyan? ho. ho.

[anon8280290]

@Guillaume

… Exposure and white balance are adjusted to known L* a* b* values of QPcard 101. <>

Step 1
I bought a ColorChecker Classic Mini and took pictures of each patch with a 0.5x magnification. Here is an example of the F3 cyan patch.
20231221_21_web.tif (2.5 MB)

Out of interest, I downloaded the linked cyan patch and measured it with the GIMP color-picker:

GIMP a*,b*: -9, -19
my card list a*, b*: -29, -29

quite a lot more than a “just noticeable difference” …

… [edit]

RawTherapee says -10, -23 …

[priort (Todd Prior)]

I think the values reported by DT are unique to the pipeline and reflect the combined output of the input profile in DT the CC and WB settings of DT…

Is there not a way in dcamprof to evaluate your profile…

As a relative indicator I’m sure DT is not bad but I think there might be a more accurate way…

[Guillaume]

@anon8280290

quite a lot more than a “just noticeable difference” …

The image you took was before calibration. I obtained a DeltaE_2000 = 1.62 with the same calibrated patch, which is a considerable improvement.

@anon8280290

As to “calibration” it is interesting that so many profile creation apps are based on the 24-patch card.

This 24-color chart is coming from an article published in 1976. It is also called the Macbeth chart, as it is written inside:

The authors are with Macbeth, Division of Kollmorgen Corporation

It has been used by many manufacturers with slight color variations. Here is a very interesting page. It has to be noted that ColorChecker changed the used values in November 2014. Many pages are still not up to date. The official cyan values of ColorChecker are 49,57/-29,71/-28,32 in L*a*b*, corresponding to 52/131/162 in AdobeRGB and -164/132/165 in sRGB, with CAT16 conversion.

@priort

I think the values reported by DT are unique to the pipeline and reflect the combined output of the input profile in DT the CC and WB settings of DT…

Is there not a way in dcamprof to evaluate your profile…

Yes, a report is displayed in DcamProf, but when you apply the profile, the values are different. You’re right, the values depend on the software used. On a perfect simulated chart in AdobeRGB (see above link), the cyan patch gives:

Gimp 2.10.36
Attributed profile = AdobeRGB
RGB = 0/133/165 → corresponding to sRGB
L*a*b* = 49.6/-29.7/-28.3 → DeltaE_2000 = 0.03

Darktable 4.6.1
Input profile = Output profile = AdobeRGB
No WB correction
With Display profile = system (56% of sRGB on my laptop)
RGB = 88/129/155
L*a*b* = 50.74/-16.26/-22.09 → DeltaE_2000 = 6.55

With Display profile = AdobeRGB
RGB = 56/132/163
L*a*b* = 49.58/-29.72/-28.32 → DeltaE_2000 = 0.01

RawTherapee 5.10
Input profile = Output profile = RTv4_medium_g2.2 (similar to AdobeRGB)
No WB correction
RGB = 56.1/132.1/162.9
L*a*b* = 49.7/-29.7/-28.4 → DeltaE_2000 = 0.14

If correctly used, these 3 tools give a good agreement with the theoretical cyan color. Gimp is suffering from RGB values in sRGB space but has correct L*a*b* values. Darktable modifies the colors measured according to the display profile. And RawTherapee suffers from none of the previous drawbacks.

Good night

[priort (Todd Prior)]

Actually it’s the histogram profile mostly for the color picker so you could check that…DT wise

[Guillaume]

I don’t want to argue but I find this behavior, the histogram and the color picker placed after the screen output, very strange. Is there a reason for that?

[priort (Todd Prior)]

No idea… if it unbounded its okay but for sure some display profiles have had issues…You can demonstrate if your profile is impacted by that order by setting both display and histogram to say linear rec2020. Your image won’t look right but now you can see if the picker values change by changing the display profile. Toggling should not change the picker value. If it does then the profile is clipping the data going to the picker… There has been talk in the past of trying to address this but apparently it’s a lot of work

[Guillaume]

Hi Todd,
I can confirm that with the histogram profile set to linear rec2020, the picker color depends on the display profile, even if it is not my screen profile.
Good night

[anon8280290]

@Guillaume

@jdc

5.8 and before Observer 2°
5.9 Observer 10° - this is what the CIE recommends
5.10 - Observer 2° - because the anomalies found were too significant…

I see, it makes sense. I suppose that the values provided by Xrite ColorChecker are using Observer 2°.

For what it is worth, ChatGPT 4 says that the colors listed with a Macbeth/Xrite card are based on the 2° Observer.