RT CIECAM02 to match colors in a photograph of a painting to colors in the painting

Elle · September 7, 2018, 2:55pm

I’m probably not the first person to use RawTherapee’s wonderful CIECAM02 module in this way. But it was a nice illustration of what color appearance models are all about. So I thought I’d share what happened when I photographed a small painting and realized the colors in the photograph didn’t match the colors in the painting.

A few days ago I painted a small “exercise in mixing foliage greens” (handprint : mixing green), in connection with topics discussed here:

To mix and paint the green colors, I used a set of 12 Mungyo water-soluble oil pastels. Similar to most such “sets of 12”, the Mungyo set provides 10 highly saturated colors plus black and white, so making believable foliage green colors did require substantial mixing.

I worked on the painting under the halogen track lighting that’s just to the right of my computer workstation. When I was done, I carried the painting upstairs to my little tabletop photography studio, where I set a custom in-camera white balance and then photographed the oil pastel painting. The lighting was mostly from a large window, augmented with an LED spot light reflected off the ceiling.

Back at the computer I opened the raw file in RawTherapee and exported a scene-referred rendition using the in-camera white balance - no use of the CIECAM02 module at this point - didn’t even cross my mind.

Then I opened the exported image with GIMP, did some perspective correction and such, and brought the actual painting back downstairs to compare the colors in the painting with the colors on the screen.

The colors on the screen didn’t look anything at all like the colors in the painting. Which upon reflection made perfect sense - I had already noticed that moving the actual painting from halogen light to window light was enough to push the greens very noticeably from warmer greens to cooler, bluer greens. I also had noticed while working on the painting that making the halogen lighting on the painting brighter made the painting itself look less saturated.

“Colors change when the lighting changes” being a major reason we have color appearance models , using RawTherapee’s wonderful CIECAM02 module seemed like just the thing to do to make the camera colors match the painting colors.

So I exported the perspective-corrected image from GIMP, opened it with RawTherapee, and put the actual painting back under the halogen lighting. Then I used RawTherapee’s CIECAM02 module to make the colors on the screen match the colors in the oil pastel, using the "Free temp+green + CAT02 + [output] option and modifying the Temperature, Lightness (J) and Saturation (S) sliders, plus setting all sliders for luminance to 18.

This isn’t saying I actually know what I’m doing when using RT’s CIECAM02 module! other than noticing that there were saturation and hue changes when viewing the actual painting colors under different lighting conditions, and so these would be the parameters in the RT CIECAM02 module to try adjusting.

For some reason I couldn’t get the cyan-blue colors in the upper right corner to match as closely as I’d have preferred. But the rest of the image is a pretty good match to the colors in the actual painting. Here’s the “before” and “after” versions of the photograph of the painting:

If anyone is curious, a larger version of the mixing exercise currently is at the top of this page (contents of the page change over time): Pictures in progress

As a side note, originally I was trying to match colors while comparing colors with different “surround” colors:

dark surround from RawTherapee’s default dark theme
white surround around the actual painting (leaning it against white watercolor paper)
image exported from RawTherapee after the CIECAM02 adjustment, and viewed in GIMP using a middle gray theme

But I couldn’t actually get the colors in the painting to match the colors as seen in RawTherapee or as reopened in GIMP until I selected RT’s gray theme and also put a more or less middle gray “surround” behind the actual painting.

afre · September 7, 2018, 4:55pm

Interesting adventure. For the past several months, I have been noticing how much surround affects the colours of my screen. The appearance depends not just on the temperature of the light source but where it is situated, how it is shone, and how it interacts with the screen material, etc. The more I learn from the people in this forum the more I realize that what is being discussed pales in comparison to the subtleties of real life.

Elle · September 7, 2018, 5:46pm

Hi @afre - I totally agree. We start out by looking at what’s on our monitors and thinking “Oh, of course that’s what the image looks like.” But as soon as we have any reason to actually look at the real world (even just looking at someone else’s monitor), or even just start to pay closer attention to the colors on the screen, all kinds of complications raise their heads. As a really basic example, I’m always amazed at how much two different colors side by side end up influence each other’s appearance.

One thing that I’ve really enjoyed about learning more about color and color appearance models is that it’s an incentive to pay closer attention to the colors out there in the real world. I’m very much more aware of changing colors as the time of day changes, or as clouds roll in and out, or etc, than I ever was before taking up digital photography and starting to poke around the edges of color appearance models.

Speaking of monitors, a weak link in my “photographing a painting and making the photograph look like the painting” is of course my monitor. I really dislike calibrating and profiling my monitor, but I think it’s again time for an updated monitor profile.

jdc · September 9, 2018, 6:55am

@Elle
My english is always bad, and the subject of the colorimetry is complex, hence probably misunderstandings.

To complete what has been said in this section I can provide some additional information.

when you know precisely :
a) the color of the subject (flower, artificial color, target to calibrate …)for which you have the spectral data
b) illuminant (daylight, blackbody…)for which you have the spectral data
c) and spectral data from the observer 2°

You can calculate easily XYZ with matrix calculation as

//calculate XYZ from spectrum data (color) and illuminant : J.Desmis december 2011
void ColorTemp::spectrum_to_color_xyz_daylight(const double* spec_color, double _m1, double _m2, double &xx, double &yy, double &zz)
{
int i;
double lambda, X = 0, Y = 0, Z = 0, Yo = 0;
double epsi = 0.00000001;

for (i = 0, lambda = 350; lambda < 830.1; i++, lambda += 5) {

    double Me;
    double Mc;

    Me = get_spectral_color(lambda, spec_color);//spectral data of the color (flower, sky, etc.)
    Mc = daylight_spect(lambda, _m1, _m2);//spectral datas of the illuminant in function of where _m1 and _m2 are calcualte in function of temp
    Yo += cie_colour_match_jd[i][1] * Mc; // obersver 2°

    X += Mc * cie_colour_match_jd[i][0] * Me;
    Y += Mc * cie_colour_match_jd[i][1] * Me;
    Z += Mc * cie_colour_match_jd[i][2] * Me;
}

xx = X / (Yo + epsi);
yy = Y / (Yo + epsi);
zz = Z / (Yo + epsi);

}

for example
//#green leaf (leskenlehti)
const double ColorTemp::ColorGreenleskenlehti_spect[97] = {
0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
0.0525, 0.0273, 0.0, 0.0378, 0.0318, 0.0164, 0.0224, 0.0276, 0.0316, 0.0266, 0.0303, 0.0290, 0.0305, 0.0286, 0.0290, 0.0303, 0.0323, 0.0323, 0.0352, 0.0383, 0.0405, 0.0482,
0.0614, 0.0743, 0.0920, 0.1015, 0.1139, 0.1192, 0.1175, 0.1216, 0.1195, 0.1145, 0.1116, 0.1009, 0.0947, 0.0867, 0.0802, 0.0754, 0.0741, 0.0709, 0.0675, 0.0625,
0.0574, 0.0579, 0.0561, 0.0565, 0.0557, 0.0511, 0.0471, 0.0419, 0.0399, 0.0372, 0.0365, 0.0395, 0.0375, 0.0382, 0.0458, 0.0555, 0.0716, 0.1002, 0.1407,
0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0
};
//
//spectral data for Daylight Cloudy 6200K
const double ColorTemp::Cloudy6200_spect[97] = {
39.50, 40.57, 41.63, 43.85, 46.08, 45.38, 44.69, 47.20, 49.71, 63.06, 76.41, 80.59, 84.77, 85.91, 87.05, 84.14, 81.23, 90.29, 99.35, 105.47, 111.58, 112.23, 112.87, 111.74, 110.62, 111.41, 112.20, 108.98, 105.76, 106.32,
106.89, 106.34, 105.79, 104.62, 103.45, 105.09, 106.72, 105.24, 103.76, 103.75, 103.75, 101.87, 100.00, 98.29, 96.58, 96.46, 96.34, 92.85, 89.37, 90.25, 91.12, 91.06, 90.99, 90.17, 89.35, 87.22, 85.10, 85.48, 85.85,
84.03, 82.20, 82.45, 82.69, 83.92, 85.15, 83.14, 81.13, 76.65, 72.17, 73.27, 74.36, 75.65, 76.95, 70.34, 63.74, 67.98, 72.22, 74.88, 77.54, 71.59, 65.65, 56.82, 47.99, 58.53, 69.06, 67.27, 65.47, 65.96, 66.44, 63.92, 61.41, 57.52,
53.63, 56.47, 59.31, 60.80, 62.29
};

After that you could manipulate datas as you want, Lab conversion, RGB conversion, etc., and of course you can “see” the differences when you change the illuminant.
But what seems obvious is that we can do this only if we know the spectral data of the colors, as well as those of the illuminant.

I used these calculations to calculate a new automatic white balance in RT. I called algoritm Itcwb (Iterate temperature correlation) with 200 colors reference.
Summarily I compare the values “xy” of the image - by varying the “temperature” and the “green”, to some of the 200 colors referenced. The choice being made from statistical computation.
The results are good, generally much better than with the algorithms based on the gray. But it is not the subject here

Other point concerning CIECAM. After many tests, ciecam only works well when using illuminants close to daylight or blackbody, that is, who have a CRI close to 100 (Color Rendering Index).
In other cases, Fluorescent, LED the results are suspicious.

garagecoder · September 9, 2018, 10:33am

As an experiment I added an “LMS adjustment” filter to G’MIC, to see what happens if you directly set the illuminant ratios. Only available with the most recent G’MIC 2.4.0. Assume no correctness, it’s merely a test (although I did try to follow CAT02).

Elle · September 9, 2018, 5:06pm

That’s a great big huge “when you know precisely”! Leastways I don’t have a clue how one would get such data short of getting a spectrophotometer and measuring everything in sight for every photograph one makes - yes? no?

Even without knowing actual spectral data, the RT CIECAM02 module is nice for getting colors “in the right ballpark”.

Awhile back there was a forum post (not in this forum) where an artist didn’t like a photograph of her artwork, even though the image on the photographer’s screen was an exact match to the actual painting sitting next to the screen. I wonder if the artist was looking at her painting under lighting with a completely different color temperature than the lighting under which the painting was photographed - it makes a difference!

When I photographed the painting I doubted the wisdom of turning on that LED spotlight, having cordially hated the color of the light it emits since the day I brought it home several years ago. But it was the quickest way to more or less even out the light hitting the painting and I was feeling lazy. The other lights for my tabletop studio are all plain old ordinary tungsten bulbs and maybe that’s a good thing even if the color temperature of tungsten bulbs is rather low.

Is this new automatic white balance code already in the RawTherapee 5.4 release? I didn’t see it, but maybe I wasn’t looking in the right place.

afre · September 9, 2018, 5:56pm

I often forget this.

I assumed that it was already integrated in the back end, that using auto WB would use it.

jdc · September 10, 2018, 6:18am

@Elle
It is not obvious to find spectral datas…I use 3 ways

a link with flowers, tree…statistics/natural400_700_5.asc at master · JelteF/statistics · GitHub
spectral datas coming from calibrating chart - I use a 468 color taget, and I have datas for another chart with 570 color, of course I have also Colorcheckers…
I use also and old (very old) software (free) from Logo Gmbh : ColorLab 2.8.13, it allows you by entering some datas in Lab values in order to obtain spectral datas for these colors…

All these datas, are with step 10nm, I interpolated them manually to 5nm!

@afre
No the actual version of white balnace auto, is always the “simple” grey one

You can find the new algorithms (Itcwb, who uses a chroma comparison), as well as others who also use gray. in a new branch “autowblocal”. You must compile , or find a windows version in RTW64NightlyBuilds/ – Keybase.pub
This version is not optimize, the team should optimize the code

jacques

Elle · September 11, 2018, 11:13am

@jdc - thank you so much for the link to the spectral data from flowers, trees, and such. I didn’t know such data was available, not that I have knowledge or expertise to use such data, but it’s nice to know it exists. How did you get spectral data for the target charts?

jdc · September 11, 2018, 11:50am

@Elle
For the 468 color chart, I build this target chart with a friend (12 years ago), to calibrate camera with high gamut values. I defined what colors in Lab mode, in what place (with limitation of the printer), then my friend build the chart, then with a spectrograh mesrured values.
We have a correspondance between Lab (and XYZ values) in D50 and Spectral datas
For example
SampleID SAMPLE_NAME XYZ_X XYZ_Y XYZ_Z LAB_L LAB_A LAB_B
END_DATA_FORMAT
NUMBER_OF_SETS 468
BEGIN_DATA
1 1 87.09 90.39 79.25 96.16 -0.13 -3.97
2 2 7.52 7.64 2.07 33.23 1.41 26.31
3 3 1.90 0.92 7.26 8.29 30.42 -47.07
4 4 3.84 3.68 18.89 22.58 4.46 -55.84
5 5 73.70 81.13 25.64 92.19 -9.18 51.06

SampleID SAMPLE_NAME nm380 nm390 nm400 nm410 nm420 nm430 nm440 nm450 nm460 nm470 nm480 nm490 nm500 nm510 nm520 nm530 nm540 nm550 nm560 nm570 nm580 nm590 nm600 nm610 nm620 nm630 nm640 nm650 nm660 nm670 nm680 nm690 nm700 nm710 nm720 nm730
END_DATA_FORMAT
NUMBER_OF_SETS 468
BEGIN_DATA
1 1 0.2278 0.2827 0.4018 0.6792 0.9131 0.9930 0.9947 0.9786 0.9699 0.9594 0.9480 0.9427 0.9357 0.9288 0.9222 0.9155 0.9084 0.9015 0.8930 0.8910 0.8885 0.8876 0.8868 0.8889 0.8905 0.8930 0.8983 0.9052 0.9093 0.9136 0.9182 0.9236 0.9297 0.9346 0.9368 0.9392
2 2 0.0138 0.0135 0.0136 0.0153 0.0165 0.0168 0.0174 0.0187 0.0202 0.0232 0.0321 0.0494 0.0664 0.0716 0.0707 0.0738 0.0769 0.0705 0.0651 0.0722 0.0832 0.0892 0.0903 0.0904 0.0926 0.0960 0.1014 0.1091 0.1171 0.1223 0.1241 0.1234 0.1205 0.1192 0.1263 0.1419
3 3 0.0166 0.0319 0.0613 0.1056 0.1366 0.1443 0.1341 0.1098 0.0827 0.0553 0.0337 0.0240 0.0185 0.0116 0.0074 0.0067 0.0061 0.0051 0.0052 0.0061 0.0073 0.0076 0.0074 0.0070 0.0067 0.0068 0.0067 0.0074 0.0084 0.0088 0.0081 0.0073 0.0068 0.0064 0.0077 0.0136
4 4 0.0160 0.0356 0.0764 0.1416 0.1964 0.2334 0.2619 0.2688 0.2544 0.2272 0.1947 0.1738 0.1517 0.1135 0.0760 0.0542 0.0353 0.0161 0.0066 0.0047 0.0044 0.0044 0.0043 0.0041 0.0040 0.0039 0.0041 0.0043 0.0047 0.0050 0.0047 0.0041 0.0041 0.0044 0.0053 0.0085
5 5 0.1021 0.1086 0.1312 0.1886 0.2318 0.2461 0.2500 0.2554 0.2629 0.2830 0.3557 0.5100 0.6855 0.7931 0.8345 0.8453 0.8464 0.8392 0.8308 0.8342 0.8386 0.8424 0.8433 0.8451 0.8467 0.8491 0.8543 0.8608 0.8654 0.8696 0.8736 0.8776 0.8825 0.8868 0.8888 0.8920

For Colorchecker, if you go deep in web, you can find values…

For the 570 color chart, it is another friend who buy an ICC profile, with a 570 color chart… To calibrate Lab and spectral datas are furnished

Colorlab also is a good tool, because you can entered values with very high gamut.