Color profile from Tiff file

Hello forum members,

I am working in a research laboratory and I want to color profile the camera sensor of an optical microscope. The camera is a Zeiss AxioCam ICc 5. The output is in czi file format,
linear tiff format, or tiff file with gamma = 1 / 2.2. I used the latter one. A ring light is present around the objective. Exposure and white balance are adjusted to known L* a* b* values of QPcard 101.

Step 1
I bought a ColorChecker Classic Mini and took pictures of each patch with a 0.5x magnification. Here is an example of the F3 cyan patch.
20231221_21_web.tif (2.5 MB)

Step 2
I then processed the gamma-tiff photos with RawTherapee using the following parameters:

  • input profile: embedded,
  • working profile: ProPhoto,
  • abstract profile: sRGB,
  • output profile: RTv4_sRGB, absolute colorimetric, no black point compensation,
  • save as Tiff 16 bits.

Step 3
The patches were assembled using Gimp giving the full Macbeth chart.
Zeiss_508_sat0_web.tif (2.7 MB)

Step 4
The resulting file was converted to linear 16 bits with Gimp and processed using DcamProf with the following Windows batch script.

"%argyll_dir%\bin\scanin.exe" -p -v -dipn -F %corners% "%target%.tif" "%argyll_dir%\ref\ColorChecker.cht" "%dcam_dir%\data-examples\cc24_ref-new.cie"
"%dcam_dir%\dcamprof.exe" make-profile -g "%dcam_dir%\data-examples\cc24-layout.json" "%target%.ti3" "%target%.json"
"%dcam_dir%\dcamprof.exe" make-icc -n "%target%_dcam" "%target%.json" "%target%_dcam.icc"

Step 5
The assembled gamma image obtained in step 3 was processed again in RT:

  • input profile: custom from DcamProf,
  • working profile: ProPhoto,
  • abstract profile: sRGB,
  • output profile: RTv4_sRGB, relative colorimetric, black point compensation,
  • adjust exposure by -0.3 EV
  • save as Tiff 8 bits.

Zeiss_508_sat0_sRGB_web.tif (2.0 MB)

Step 6
Compare obtaind L* a* b* values to theoretical values and calculate ΔE_2000 color difference. The values are as follows:

  • original file: 9.90 and 13.52 for A1 (dark skin) and F3 (cyan) patches, respectively,
  • corrected file: 5.21 and 3.31 for A1 (dark skin) and F3 (cyan) patches, respectively.

Conclusion
Using this protocol, I was able to obtain improved color values. However, the result is not satisfying. In particular, the luminance axis seems to be contracted. How can I improve it ?

Thank you for your help and your insights.
Guillaume

PS: The above photos were resized using XnView MP to 1024 pixels.

Hello,

In addition, I used the ColorChecker simulation prepared by BabelColor in Adobe RGB 16 bits. I just applied the Adobe color profile under Gimp and saved the file. All the L* a* b* values were as expected.

However, under RawTherapee with neutral profile, which means:

  • input profile: embedded,
  • working profile: ProPhoto,
  • abstract profile: sRGB,
  • output profile: RTv4_sRGB,

we obtain the following values for the F3 cyan patch: L* a* b* = 50.8/-22.6/-26.2 instead of L* a* b* = 49.57/-29.71/-28.32 in theory, giving a ΔE_2000 = 3.38. This is rather bad.

Am I missing something? @ggbutcher @KatrinaBSLPrints any idea?

Hello Guillaume,

I stumbled onto your post by accident, looking for TIFF embedded profiles. The issues that you describe seem very familiar. I will think about it for a bit but rest assured that probably 75% of all prepress professionals don’t ever dive in as deep as you want/need to. This also explains why colors are (still) an absolute s***show, and now add that Pantone/X-Rite is the only company worldwide who sells this and they are worse than Microsoft (in my opinion).

Getting a ColorChecker was definitely a good idea. I’m trying to approach this from the top because I want to fight the cause, not the symptom :slight_smile: if that’s alright with you.
Do you know the exact type/name of the ring light? Do you know about anyone that used the device and cared about color, besides you? The internet says it’s quite an old lady - I mean, Firewire… that’s one step up from SCSI!

All the best
Rafael

PS: Does this forum have PMs!?

Sorry for the delay, vacation travel.

So, you’re comparing a sampled patch value from a rendition exported from GIMIP in AdobeRGB with the same renditon exported from RawTherapee in sRGB… if the patch is an extreme color I wouldn’t expect the *a and *b values to be the same.

Hi guys,
Thank you both for your answer. I was feeling myself a bit alone :wink: .

Do you know the exact type/name of the ring light? Do you know about anyone that used the device and cared about color, besides you?

I found the manual but I don’t remember the name. The optical microscope is a Zeiss Stemi 508. The ring light is very specific for this device, with LED all around the sample. I can adjust light intensity as well as exposure time for correct exposure. When measuring a gray patch on the image, the obtained white balance is 6500 K. I just realized that the light is polarized, and parallel or perpendicular polarization is easy to select. The latter strongly decreases glare on ColorChecker patches, as expected due to filtering.

I know people using this device, but nobody cares about colors. Even the seller has no idea of a color profile.

Sorry for the delay, vacation travel.
So, you’re comparing a sampled patch value from a rendition exported from GIMIP in AdobeRGB with the same renditon exported from RawTherapee in sRGB… if the patch is an extreme color I wouldn’t expect the *a and *b values to be the same.

No problem! I hope you had a good time during your vacation.

This scheme is very informative. Correct me if I am wrong but I think that the input profile tries to revert to the recorded luminance. The abstract profile then applies the encoding transfer function which is then converted to the output profile.

Following your suggestion, I checked with Input = AdobeRGB1998.icc, Abstract = Adobe g 2.2, and Output = RTv4_Medium. Thus all of them are Adobe color space. With the ColorChecker simulation, I obtained L* a* b* = 53.3/-15.2/-22.4 for the F3 cyan patch, giving ΔE_2000 = 7.92. According to https://wisotop.de/assets/2017/DeltaE-%20Survey-2.pdf , these are very different colors.

I don’t understand why the result with Adobe Color space is so bad.

input (camera) profiles are almost always linear, no tone transform. Their primary purpose is to give a starting point for the subsequent color transforms to working and rendition color gamuts. Matrix camera profiles are a coarse representation of the camera spectral sensitivity, only nine numbers to represent a rather rich topography. Adobe DCPs expand on that fundamental purpose with LUTs and sometimes curves to impart a “look” in the initial transform, a bit of confusion in my opinion.

Just to illustrate their place in the process, my raw workflow doesn’t do any color transform until export or display, with all the raw processing done on the data in its original camera color and linearity. If I do a TIFF to go to GIMP or some other software for further work, I’ll convert to some working profile like ProPhoto or Rec2020, mainly because I don’t trust the other software to be kind to camera data…

Thank you for your help @ggbutcher. I read your article and a few others. I understand your point about RAW files. However I have only access to TIFF files. My problem can be separated in two questions:

  1. How can we import a TIFF file with correct L* a* b* values knowing the color profile, either embedded or not?
  2. In the particular case of my optical microscope, I need to create an icc or dcp profile and I suspect there are some errors in my process, especially due to glare. Could you check?

Update: I just upgraded from RawTherapee 5.9 to 5.10 under Windows 11 and L* a* b* values are very different, using the same process. Has color management changed under the hood? Nevertheless using Adobe abstract profile still degrades the color accuracy obtained with none or sRGB, which seems unexpected for me.

  • A1 dark skin: RT_5.9 gives 36.1/11.3/13.4 (ΔE_2000 = 2.6) → RT_5.10 gives 37.5/14.7/14.6 (ΔE_2000 = 0.4), expected values are 37.54/14.37/14.92;
  • F3 cyan: RT_5.9 gives 52.0/-18.3/-24.6 (ΔE_2000 = 5.8) → RT_5.10 gives 49.7/-29.8/-28.2 (ΔE_2000 = 0.2), expected values are 49.57/-29.71/-28.32

@Lawrence37 @martbetz @jdc, your insight will be appreciated.

Here are the screenshots.
Regards,


Hello Guillaume

I will have a look, tomorrow.

I think your are french ? If so…

Mais, je ne garantis rien…Les derniers travaux que j’ai réalisés sur ce sujet datent de 2007…et je n’ai plus de données.
Néanmoins voilà ce que je disais… il y a 11 ans
https://rawpedia.rawtherapee.com/Color_Management_addon/fr#Les_profils_ICC_%22input%22_:_%C3%A9laboration,_utilisation,_lacunes

Mais promis, je regarde… au moins pour comprendre la demande.

jacques

Bonsoir Jacques,
Merci pour ce partage. En effet, je suis Français. Si besoin nous pourrons en discuter par téléphone.
Bonne soirée

Hello Guillaume

I will speak in French, the subject is sufficiently complex, not to add a language that I do not master well.
But I will do a translation so that the forum can follow.

==========
Sujet oh combien complexe. En temps normal, pour élaborer un profil ICC on part d’un fichier raw “étalonné” . A la fin avec le TIF produit par “Save Reference Image” (onglet Color). Avec ce TIF et les données de la mire (ici la colorchecker24) on élabore un profil ICC qu’on applique ensuite à toutes les images que l’on souhaite “traiter”. Ce profil est appliqué très en amont dans le processus. Tout ce traitement se fait en mode linéaire, sans gamma.

Ici pas de Raw, mais un TIF, d’où des difficultés qui apparaissent. Car une fois l’image obtenue et un profil créé …où le mettre ?

1ere solution:
Examinons le processus que tu décris. Tu as choisi a priori la sortie en TIF gamma=2.2, pourquoi pas, mais le linéaire devrait être (probablement ?) meilleur.
Mais quel est l’illuminant (l’éclairage du microscope), point important car selon celui-ci le CRI (Color Rendering Index) sera plus ou moins bon…et pas moyen de corriger.

Ensuite la balance des blancs, en TIF, les moyens de contrôle sont (très) limités. Mais tu peux utiliser dans “White Balance” - Pick - sur une des cellules grises de la Corchecker24. Regardes les caractéristiques et choisis celle qui les valeurs “a” et “b” en mode Lab les plus petites. La cellule 22 semble “la moins mauvaise”.

A priori ne pas mettre de correction d’exposition, car elle intervient “après” dans le pipe-line.
Si tu n’as pas moyen de changer à la prise de vue…difficile…

Ensuite, pas de conversion vers Adobe ou autre chose, pas d’utilisation de “Abstract profile” (au moins dans cette hypothèse). Mais utilise “Save Reference Image” (onglet Color).

2ème solution qui est complètement différente de celle-ci. Ceci suppose que tu te serves de Rawtherapee pour regarder, traiter tes images.
Utilise le processus que je propose dans Rawpedia en se servant de “Abstract Profiles” et Ciecam (Color Appearance & Lighting)
https://rawpedia.rawtherapee.com/Color_Management#Possible_example_of_use_to_modify/improve_the_behavior_of_the_Input_profile
ou le même en français
https://rawpedia.rawtherapee.com/Color_Management/fr#Exemple_possible_d’utilisation_pour_modifier_/_am%C3%A9liorer_l’action_du_profil_d’entr%C3%A9e

Je pense que la deuxième solution, et l’utilisation de “Paste partial” “Color management”, devrait répondre (peut être ? ) à tes attentes.

Je vais te passer un message perso pour que tu aies mes coordonnées.
Jacques

=====
Oh so complex subject. Normally, to develop an ICC profile we start from a “calibrated” raw file. At the end with the TIF produced by “Save Reference Image” (Color tab). With this TIF and the data from the target (here the colorchecker24) we develop an ICC profile which we then apply to all the images that we wish to “process”. This profile is applied very early in the process. All this processing is done in linear mode, without gamma.

Here no Raw, but a TIF, hence the difficulties which appear. Because once the image is obtained and a profile created…where to put it?

1st solution:
Let’s look at the process you describe. You chose a priori the TIF output gamma=2.2, why not, but the linear one should be (probably?) better.
But what is the illuminant (the lighting of the microscope), an important point because depending on it the CRI (Color Rendering Index) will be more or less good…and there is no way to correct it.

Then the white balance, in TIF, the means of control are (very) limited. But you can use “White Balance” - Pick - on one of the gray cells of the Corchecker24. Look at the characteristics and choose the one with the smallest values “a” and “b” in Lab mode. Cell 22 seems “the least bad”.

A priori, do not use exposure correction, because it occurs “after” in the pipeline.
If you don’t have a way to change when shooting…difficult…

Then, no conversion to Adobe or anything else, no use of “Abstract profile” (at least in this hypothesis). But use “Save Reference Image” (Color tab).

2nd solution which is completely different from this one. This assumes that you use Rawtherapee to view and process your images.
Use the process I propose in Rawpedia using “Abstract Profiles” and Ciecam (Color Appearance & Lighting)
https://rawpedia.rawtherapee.com/Color_Management#Possible_example_of_use_to_modify/improve_the_behavior_of_the_Input_profile

I think that the second solution, and the use of “Paste partial” “Color management”, should meet (maybe?) your expectations.

I will send you a personal message so that you have my contact details.

Jacques

1 Like

@Guillaume

J’ai oublié ta remarque sur la différence entre les 2 TIF - 5.9 et 5.10

Je ne vois qu’une raison possible, le changement de “Observer” qui sert en colorimétrie avec l’illuminant et les couleurs de base de l’image.

5.8 et avant Observer 2°
5.9 Observer 10° - c’est ce que recommande la CIE
5.10 - Observer 2° - car les anomalies trouvées ont été trop importantes… Normalement il y a “compatibilté”… Mais dans ton cas, puisque tu commences de rien, pas d’importance.

=========
I forgot your remark on the difference between the 2 TIFs - 5.9 and 5.10

I only see one possible reason, the change of “Observer” which is used in colorimetry with the illuminant and the basic colors of the image.

5.8 and before Observer 2°
5.9 Observer 10° - this is what the CIE recommends
5.10 - Observer 2° - because the anomalies found were too significant… Normally there is “compatibility”… But in your case, since you are starting from nothing, it doesn’t matter.

Jacques

Hello @jdc,
I am staying in English so other people can comment on our discussion. Thankfully, deepl is our friend :wink: .

I see, it makes sense. I suppose that the values provided by Xrite ColorChecker are using Observer 2°.

The default encoding gamma is 0.45, which corresponds to the invert of 2.2. I did quite a lot of tests with linear TIF. Results were bad with RT 5.9 but I can try again with RT 5.10.

The illuminant temperature is 6500 K with LED. White balance and exposure are adjusted before the picture shot, using a QPcard 101 black/gray/white target. All the set-up is warmed up for 15 minutes to avoid color shifts.

The idea is indeed to use RT to process an entire batch of photographs taken with the exact same parameters. However, I am not the only user of the optical microscope and the process has to be as simple as possible, typically using a saved preset.

Your solution is interesting. Will it be as precise as using an ICC or DCP profile?

Thank you so much, your answer is very helpful :+1: .

Hello Guillaume

Nothing prohibits combining the 2 approaches.

The first via an ICC or DCP profile seems more “rational” (which is completely false), it is simply more usual.

The instructions for sharing “partial paste” may be a little more complex, in that it will require an additional step (this is only a learning problem).

As far as precision is concerned, it is only a question of patience and doing it once, modifying the primaries and the TRC (Tone Response Curve) once, to obtain the desired results. I think (I believe) that the differences in terms of deltaE (maybe I am wrong), must be minimal.

Perhaps, not sure, to check that using the “Local adjustments” with the deltaE (we are not talking about the same thing as Colorchecker, but about the algorithm that I developed), should allow a better final rendering with the use of what is currently in the “lacam16n” branch. Indeed, with several RT-spots we must be able to target more precisely the adaptation of the primaries (and the TRC) to each color range. After more rethinking, I think I went too fast, it must not be able to work with.
Executables - Pull Request - lacam16n
https://github.com/Beep6581/RawTherapee/releases/tag/pre-dev-github-actions

I admit that “Abstract profiles” or its equivalent in LA are not conventional approaches (to put it mildly), but it works.

Jacques

5.10 fixed a long-standing bug in which the lockable color pickers and the navigator showed incorrect L*a*b* values (Issue #5408).

2 Likes

Hello Jacques, hello to all
It was a real pleasure to discuss with you by phone. Thank you for your advice. Here is an update on my work using RT 5.10, as this version corrects L* a* b* bug. Sorry for the delay, it’s not my only project.

ICC profile creation
1) Reference image with RT
Starting with BabelColor Macbeth chart, I saved it as a reference image. It’s identical to the original image, still a gamma-encoded one and not a linear one.

2) Linear with RT
input profile: AdobeRGB; abstract profile: linear; output profile: Prophoto. The resulting image was processed with DcamProf.

scanin.exe -v -p -dipn -F %corners% target.tif ColorChecker.cht cc24_ref-new.cie
dcamprof.exe make-profile -g cc24-layout.json target.ti3 target.json
dcamprof.exe make-icc -t linear -W -n target_dcam target.json target_dcam.icc

Here is the profile displayed with GamutVision, in dotted line. The plain line is the AdobeRGB profile. As you can see, they are strongly different.
gv_lin_RT

3) RT with linear Gimp
input profile: AdobeRGB; abstract profile: none (=sRGB); output profile: RTv4_medium. The image is then converted to linear using Gimp. There is almost perfect agreement between the two profiles.
gv_nometa_RT_lin

4) DcamProf options
Adobe RGB and its equivalent RTv4_medium use D65 as the illuminant. Adding this option to make-profile induces a shift of green vertex to the right (not shown).

dcamprof.exe make-profile -i D65 -g cc24-layout.json target.ti3 target.json

acr option is often suggested for make-icc. The result is shown below, clearly degrading the profile.

dcamprof.exe make-icc -t acr -W -n target_dcam target.json target_dcam.icc

gv_nometa_RT_lin_acr

ICC profile usage
Input profile: the one obtained in step 3 above; abstract profile: linear; output profile: RTv4_medium. L* a* b* values are as follows:

  • A1 dark skin: 37.2/13.6/13.5 (ΔE_2000 = 0.95), expected values are 37.54/14.37/14.92
  • F3 cyan: 46.8/-35.9/-31.0 (ΔE_2000 = 3.95), expected values are 49.57/-29.71/-28.32

Color accuracy is excellent on dark skin patch. Cyan patch is not as good but is expected to be difficult as it is outside of the sRGB color space. Next step is to check this protocol with a TIFF obtained from a flatbed scanner. Finally, I will be able to apply it on my microscope images which are very similar. I’ll keep you in touch.

1 Like

I neglected to mention that when profiling, turn off the abstract profile. It leverages the same tools used for color management, but is intended for applying creative effects.

1 Like

In french first :

Bonjour Guillaume

Je suis vraiment content que les échanges d’informations lors de la vidéo de la semaine dernière, t’ont permi d’arriver à obtenir un résultat acceptable. Comme je te l’ai dit, peu importe la manière (calibrer un TIF n’est pas courant), l’essentiel est que tu sois satisfait du résultat.

Les résultats de deltaE sont tout à fait acceptables, le cyan est un “poil” limite, mais c’est correct.
Tiens nous au courant des suites Ă©ventuelles.
Merci

And in english

I’m really pleased that the exchange of information in last week’s video enabled you to achieve an acceptable result. As I said, it doesn’t matter how you do it (calibrating a TIF is not common), the main thing is that you’re happy with the result.

The deltaE results are quite acceptable, the cyan is a “hair” borderline, but that’s okay.
Keep us posted on any further developments.
Thanks

Jacques

Hi @Lawrence37 ,

What do you mean by turning it off? Is it the linear value? I am surprised that the default is falling to sRGB. I understood the process like this, but correct me if I am wrong.

  • input profile: convert to linear,
  • abstract profile: apply gamma + creative effects,
  • output profile: define primaries,
  • rendering intent: dilatation/compression/clip to the output profile.

Thanks.

Something is weird - the default is “None”, or at least was for a while. I haven’t recompiled in a while, but it’s definitely “none” on my fairly old build.

You may find WIP: Bundled profile for reference image by Entropy512 · Pull Request #6646 · Beep6581/RawTherapee · GitHub somewhat useful - apologies for not finishing this one prior to 5.10, I was catastrophically burned out last year and I’m still recovering from the mental health issues a new supervisor at my old job caused.

I would strongly recommend working with linear tiffs if possible… Possibly the reason you might have had undesirable results is that these tiffs might not be properly tagged with an ICC profile, and RT/gimp might be interpreting them as gamma-encoded when they’re not. Can you give an example file of a linear TIFF export so we can check the color profile?

Also, you had your working profile and output profile differ - you either want to use RT’s reference image function (which won’t properly tag the output, but will disable almost all color management), or set the working profile to be the same as the output and input profiles so that all color management transforms are a no-op. (see my pull request)

Sorry for not fully reviewing everything so far - I’m traveling at the moment and just noticed this.

Edit: OK, after rereading the steps you had for RT were for a processing run after profile generation, not a profiling run. Makes more sense, but I agree that I would avoid using abstract profile. I still myself haven’t figured out what a valid use case is for it, and unfortunately, there have been frequent mistakes where people set it thinking it was necessary but it caused problems, such as Reference TIFFs are not tagged with an ICC profile that indicates linear data · Issue #5575 · Beep6581/RawTherapee · GitHub

Can you provide an example linear TIFF file from the microscope? It looks like the gamma-2.2 TIFF you posted has no ICC profile attached, if the linear ones lack a profile that’s gonna be a big problem unless you add an ICC profile using exiftool, or have software that lets you override the input profile to be linear. But in general linear is usually a better choice here. I suspect using gamma encoded images is part of your problem - especially if Zeiss used a plain 2.2 gamma, which isn’t actually what sRGB is - it’s a mix of a linear segment and gamma 2.4, which “averages out” to around 2.2 - sRGB - Wikipedia

1 Like

Hi @Entropy512,
You’re right, the default is none in RT 5.10 but the result is equivalent to sRGB, that’s why I was saying it was falling to it.

Thank you, I will have a look. Mental health is a real difficulty and I hope you are recovering well.

Here is one of the photographs in linear TIFF. This is a QPcard 101 chart. The expected L*a*b* values are 93/0/0, 47/0/0 and 34/0/0. As you can check, no color profile is embedded in the metadata. Here is below the output of exiftool.

exiftool
PS > exiftool.exe .\20230929_01_ORG.tif
ExifTool Version Number         : 12.76
File Name                       : 20230929_01_ORG.tif
Directory                       : .
File Size                       : 30 MB
File Modification Date/Time     : 2023:09:29 17:04:56+02:00
File Access Date/Time           : 2024:02:26 17:31:38+01:00
File Creation Date/Time         : 2023:12:13 10:54:02+01:00
File Permissions                : -rw-rw-rw-
File Type                       : TIFF
File Type Extension             : tif
MIME Type                       : image/tiff
Exif Byte Order                 : Little-endian (Intel, II)
Subfile Type                    : Full-resolution image
Image Width                     : 2452
Image Height                    : 2056
Bits Per Sample                 : 16 16 16
Compression                     : Uncompressed
Photometric Interpretation      : RGB
Image Description               : 20230929_01
Strip Offsets                   : (Binary data 17744 bytes, use -b option to extract)
Samples Per Pixel               : 3
Rows Per Strip                  : 1
Strip Byte Counts               : (Binary data 12335 bytes, use -b option to extract)
X Resolution                    : 300
Y Resolution                    : 300
Planar Configuration            : Chunky
Resolution Unit                 : inches
Software                        : ZENcore
About                           : uuid:faf5bdd5-ba3d-11da-ad31-d33d75182f1b
Creator Tool                    : ZENcore
Title                           : 20230929_01
Description                     : 20230929_01
Rating                          : 0
Date/Time Original              : 2023:09:29 12:58:24
Create Date                     : 2023:09:29 12:58:24
Sub Sec Time Original           : 00
Sub Sec Time Digitized          : 00
XP Title                        : 20230929_01
XP Comment                      :
XP Subject                      :
Image Size                      : 2452x2056
Megapixels                      : 5.0
Create Date                     : 2023:09:29 12:58:24.00
Date/Time Original              : 2023:09:29 12:58:24.00

I need both: as there is no embedded profile, I first need to generate a profile with a ColorChecker and second to process other pictures taken in the same conditions.

That’s why I am applying AdobeRGB as the input profile, which has a pure gamma of 2.2 without linear part, corresponding to the inverse of the encoding with 0.45.

Regards