match JPEG colors in Darktable

I tried a bit too using something… I think it was called dcp2icc you can also break it down to a json file in dcamprof and the write it out but the dcp were d65 and dual illuminant and icc were d50 single illuminant and I don’t think you would get the same corrections in the icc from the hue sat tables… I was never able to land on a combination that seemed to create anything too useful… but thats my fumbling… To be honest from my experience messing around …DT-chart or the colormatch script from Pascal ( GitHub - pmjdebruijn/colormatch: ColorMatch ) were the best two in my hands to get something out of the gate that was matching the jpg… Personally I can live with just editing to taste…

1 Like

If you consider the hassle of making a glare-free target shot, converting the DCP to ICC is rather simple. Here’s what I did with dcamprof, using the Adobe Standard DCP for my Nikon D7000:

$ dcamprof dcp2json "Nikon D7000 Adobe Standard.dcp" nikon_d7000.json
$ dcamprof make-icc nikon_d7000.json nikon_d7000.icc

That sounds a bit like my experience - I did get a resulting ICC profile but it wasn’t “right”. I ended up settling for a custom profile to be accurate as opposed to jpg-matching.

But I’m well aware that this whole field is something that I’m not too worried about - so maybe I should shut up :sweat_smile:

Well, it looks like it… in your example! My result was distinctly odd in the colours… and I think I used exactly what you show… maybe I messed up somewhere.

Mine changed the colors also, why I never went past just figuring out how to ICC them. I have spectral profiles for all my cameras, but I just use the libraw-provided matrices for most processing, rendered colors are pleasing enough for me…

1 Like

I know that you have done a lot of tinkering and your work with the spectral stuff gives you great insight I am sure so nothing I say will be new to you and it might not even be correct but I think with darktable this issue is that people just need to settle on a white balance approach and then next on a color approach. For example you can use the legacy WB… read from your camera data and or set by selecting a neutral area. Traditional and fairly straightforward, ie that is a “white” balance. Or you can use the CAT method… which is as I understand it more of a neutral balance than a strict scaling of rgb to make white be white. The role of the CAT is to account for the perceptual aspect of color and to make things that were neutral in the scene appear neutral on the display device or even color appear the same. The result is often very similar to WB but not always. This seems to throw people and have them embark on endless journey to sort it out. The final variable and often something that is a problem is that the CAT calculation for this is based on a set of D65 reference values that have been shown to be off for some camera’s and so they can alter or impact the result.

THe CAT may be a “newer” approach or alternate one but just because its newer people don’t need to force feed it to themselves and I don’t think DT is severely handicapped if you don’t. You can mask it and this can in some circumstances be a benefit for complex images but in the end using legacy vs CAT for most images is not a handicap in my mind… I guess what I am trying to say is that the CAT is a different approach to introduce white or neutral balance to your image its not simply by default a better WB because its newer or more complex…

At least this is how I came away thinking about it from this…

Chromatic Adaptation explained.pdf (3.7 MB)

In any case getting wb sorted in your workflow is going to be the gateway to then work on getting colors that are either accurate or providing a look whatever the user is after…

1 Like

Oh, I agree completely. If the white reference isn’t set properly, there’s no way one can assess colors, either colorimetrically or aesthetically.

I like to use this exercise to demonstrate what white balance is combatting:

In the toolchain in the top-left pane, note there’s all the normal processing except white balance is turned off. This is not about temp/tint…

1 Like

Count me in that camp.
I did convert profiles from Adobe Camera Raw and Capture One to find something better for the Sony A7iv, but it didn’t work out. I think it may not only have to do with the conversion of the profiles, but also with the the input darktable is feeding into the resulting icc profile.

Anyways, I ended also up with a color checker and a custom profile.

1 Like

@Tamas_Papp, I’ve just tried making an ICC profile for the GX9 from this shot here: https://www.imaging-resource.com/PRODS/panasonic-gx9/GX9hVFAI00100.RW2.HTM

Assuming this is a good profile shot (not sure but it seems reasonable), using this matrix profile as the input profile in DT should (arguably…) do away with the need for a matrix in color calibration. (For purposes of accuracy, that is.)
GX-9from Imagingresourcesiso100shot.zip (398 Bytes)

Color calibration can still change colours though - especially if the D65 WB reference value is off as @priort mentioned above. And, again, the this profile is trying to be accurate, i.e. true to life which may not be the same thing as the Jpeg… this is my approach anyway!

Now to test it… I downloaded an image (raw and jpg) from the Imaging Resource (https://www.imaging-resource.com/PRODS/panasonic-gx9/Y-P1011255.HTM), and opened it up in dt. First, I notice the slight reddish tone you mentioned. (BTW I am using the ‘legacy’ WB module only workflow)
Iit turns out that my new profile doesn’t make much difference - suggesting that dt’s standard input profile is pretty good.
Using color calibration in the default manner with the WB module set to reference slightly increases the reddish cast but not substantially.

Next I tried using the same colour checker shot to set up a calibration preset for CC - not much better. I think that target shot might not have been shot under daylight - but in any case it didn’t help the reddish look either.

Next try - an ICC profile using the LUT to provide more accurate color matching than just a matrix. This sort of breaks the scene-referred workflow but tried it anyway. Still that reddish tone!

I’m inclined to think that the GX9 is behaving similarly to my EM5ii - just getting an apparently accurate profile doesn’t produce very pleasing colours. :man_shrugging:
Actually, I think it’s looking quite good on the test shot, but it’s definitely different to the jpg.

Next stop I want to try matching the jpg by eye as has been suggested!

1 Like

@Tamas_Papp

Ok… if you like, try this style?
sg123-matchjpgGX-9.dtstyle (4.2 KB)

On my example shot (as above in last post) this is looking quite good. Set the WB module to as shot and apply the style. Make sure Filmic is off, and that CC is set to bypass after applying it. I think got it right in-style but just to make sure!
Using sigmoid here, mostly as it looks closer to the jpg with hue preservation turned right down to zero.

2 Likes

Thanks for looking into this. Using studio test images from various sites is an excellent idea, this did not occur to me before.

The style you attached has a greenish cast on my yellows though. So I downloaded the test image from dpeview, their “daylight” version has a 5500k illumination, so I had

  1. white balance: camera reference
  2. color calibration: as shot
  3. color calibration (2nd instance): calibrated using the color chart.

The preset is attached.
GX9 _ dpreview.dtpreset (1.1 KB)

My output ΔE is 1.78 and with a max of 4.55, I am not sure if that is large or small, if someone can put this in context I would appreciate it.

It removes a lot of red and green from adjacent channels. I put it before the default color calibration, hoping that all the module does is a matrix multiplication which is commutative (it seems to be). Then it looks like a reasonable basis for a correction. Now I know what to look for I can tweak it is needed.

I was afraid it might not be right on all images - that style was a purely ‘eyeball’ effort. Looked good on the one, but I’ve found before that that’s not a good guide.

I’m certainly not an expert, but this page in manual might be of help darktable 4.4 user manual - color calibration

I think your results seem ok, although not 100% sure about the max. It will be a certain color patch that ill be less accurate than the others I think.

Thanks, the manuals says that

ΔE < 2.3 means that the average observer will not be able to tell the difference between the expected reference color and the obtained color. This is a satisfactory result.

so I am good.

Now that this is resolved, back to taking photos :wink:

@TonyBarrett, if you still use a GX9, you may be interested in the correction I attach above. Please let me know how it works for you.

2 Likes

Try running DT chart and see what you think about the result… The funny thing is when I play with these things ie icc or CC corrections and watch the vectorscope there are changes but many appear to be subtle wrt change in acutal hue… Looking at the Gx9 on the vectorscope… cyan is rotated massively to blue and magenta a lot towards red… yellow a bit towards red and green towards yellow… red and blue seem bang on and the jpg is not markedly different wrt hue angles so its not undergoing a massive change there… It may be that you cant do something like you see in Davinci and line up these primaries on the vectorscope or maybe there is some nuance to the vectorscope presentation but for sure this shift or alignment of magenta and cyan would impact the nature of red the camera shows…

You can see it here with the as shot wb

Setting it to the camera specific daylight from the meta shift WB quite a bit but actually aligns the hues a bit better swinging yellow green and magenta…

In any case to me that pattern on the vector scope is the GX9 color signature of sorts and I would that thought that calibrations might have move the needle more that it seems to …

1 Like

They get marked so you can see the ones that are less accurate or in the “fail” zone…

1 Like

Well really you are not. The approach just has to be a bit different… Now you would select none and then use autopresets to apply whatever combination of modules you want. If the module does something on startup then you just add reset to the preset so that it applies in its default state…so you could do this will filmic as well for example or others that make small adjustments when they are first initialized…

image

1 Like

Both darktable and RawTherapee have an black level offset, compared to Exif. For RawTherapee it is Exif (128) + 16. darktable has it locked to 143. Increasing this to 144 or 145 makes the image less red.

If you want to calculate your black level GitHub - lightful/hraw: Hacker's toolkit for image sensor characterisation and a black raw file should work. See here for more about black level and hraw Measure black and white levels?
In the sample 1Ds, like your GX9, doesn’t have any hidden black fields to analyze. Therefore a picture with lens cap on should work.

2 Likes

If you have a calibrated monitor, it’s easy to create custom WB coefficients. It’s probably explained in the manual, but if not, AP covers it here:

The DataColor Spyder Checkr Photo can often be found for quite cheap, and DT added support in the last release.

1 Like


GX9hVFAI00100.RW2 (23.2 MB)
G9jpg.dtstyle (5.0 KB)

So this is the sort of thing you get from DT-chart…
I made it simple… I exported the images attached above as needed to pfm files with as shot WB no CC.
I ran the process matching to the jpg…
Open the raw using only as shot WB and EV to set white patch to 92 using the LAB picker…
Then applied the style… It really has two main components a tone curve and the clut module.
To my eye it does a really nice job of the blues to spread them out and lighten them… I think red looks better as well…

So this style could be dissected a bit and used as a color adjustment without the tone curve if one objects…using other modules for the purpose… You could try to make it using CC in the source images as well… Its a nice tool but older so people don’t use it

This is the same but matched against the colorchart values not the jpg…

GX9ColorChecker.dtstyle (5.0 KB)

1 Like

Thanks! I tried that and

→ ./hraw rgbstats -i P1002224_0.pgm                                                                                                                                                                                                                                               [58061bd]
width;height;X;Y;;;;;;;;;;;;;
2600;1952;0;0;;;;;;;;;;;;;

R mean;R min;R max;R stdev;G1 mean;G1 min;G1 max;G1 stdev;G2 mean;G2 min;G2 max;G2 stdev;B mean;B min;B max;B stdev;
142.394;131;152;1.36246;142.827;134;189;1.30148;142.884;134;230;1.29193;142.454;132;177;1.35626;

so maybe the 143 (the default in Darktable at the moment) is not that unreasonable?

It seems good. You can verify so that it is the same at high ISO also.