I’ve done a bunch of snapshots , and it is clear the ones that ‘go bad’ are the ones where the lighting conditions where tungsten / lowlight / indoor.
Anything close to daylight or with a flash is just fine.
If i quickly disable color calibration , and then judge the white balance starting point (what looks better , reference / daylight or tungsten ?). So i just use the tungsten preset in those cases. Then I enable color calibration and I can sample or detect the correct white balance and it looks fine. I just have to ignore the warning then that the color calibration gives me about ‘double white balance set’.
Basically , if the white balance in ‘reference’ looks worse then white balance in tungsten, color calibration can’t set the colors correctly. So i just need to quickly judge which starting point seems better. Doesn’t has to be perfect , but has to be closer to 'looking somewhat correct '.
Whenever you do such an exercise, you need to include a white reference in the scene. What the presets do is just groping in the dark, so to speak; taking a measurement patch from that white reference will be the only true baseline you can carry away from the scene itself.
And then, you really need to situate that white reference under the light you’re most interested in characterizing. Daylit interior can be a mess depending on time of day and the component reflectances of all the surfaces in the path from the light to the subject.
I know nothing about how darktable’s color calibration works, but it would seem from your poking around that it has some sort of D65 dependence… ???
Well, you can’t always have a gray or white reference in the frame. It’s not like every shot is planned beforehand or something.
Also, it’s not about getting a good color balance in the end. It’s about the difference between legacy (just white balance module) or modern (white balance module set to a reference ‘default’ and then color-calibration to get the colors right).
In the manual, it’s said that you need a ‘reference’ white balance to make demosaicing perform OK. But that has nothing to do with quite different end results (and the fact that I have the same effect with already demosaic’d linear-DNGs).
I think it has to do with what the input-profile for the camera does. The order of the pipeline is ‘white balance → input profile → color calibration’. So if the input profile does something weird with the colors, color calibration can get confused. And I think this is what is said in the manual about Sony and Olympus cameras.
Their input profiles ‘fix quite a lot’ apparently, which means that if your starting color before the input profile is not ‘correct enough’, the input profile will only throw them further out. And at the end, the CAT in color calibration can’t make heads or tails about what the perceptual situation would be.
So I don’t think ‘color calibration’ has a dependency on D65. I think the Sony input profile has a dependency on ‘good white balance in’, where other cameras apparently don’t. The fact that color-calibration seems to behave nicely when the input-profile is set to linear-rec2020 (so basically a no-op, if the working profile is also linear rec2020?).
For the record, rawtherapee’s CAM module also says that the white balance needs to be ‘a technical correct white balance’ before using the module. I guess this is similar to the ‘reference’ white balance Darktable uses.
I also have the same problem occasionally. I think I even mentioned it once in another thread, thinking it was related to the colour preservation of filmic.
It is clear to me that ideally one should carry a neutral grey reference. But it’s just as clear that this is the exception when you’re not working in a studio or doing professional shootings.
At the moment, I simply switch off the colour preservation in film. It’s certainly not the cleanest solution and other colours shift a bit, but if the main subject are people, pleasing skin tones are more important.
It would be great, if there’s a better (or more correct) solution.
Did I understood it right that color calibration never reads the coefficients reported in the exif data?
The problem I’m describing is with filmic disabled , so that’s not cmy problem. Never had a problem with filmic colors to be honest. Discovering filmic in 3.4 and 3.6 made me switch to Darktable to be honest.
No, i do not think that’s right . Color calibration has a ‘as shot’ mode, just as white balance has. It’s even the default when loading a raw file .
Color calibration comes after demosaicing, exposure , camera profile … That way it can preset an accurate representation of the colors taking scene and lighting into account . And in my case the camera profile seems to make a mess of things if the white balance isn’t set ‘sort of ok’ yet .
White balance comes before demosaicing and everything. Since demosaicing needs to make choices about where to interpolate which color and what kind of detail , having a sort of ok white balance set helps .
So the modern color workflow in Darktable is to white balance with a fixed reference white balance (something like daylight preset for your camera ) , demosaic , do exposure and apply camera profile, and then finally do white balance ‘for real’, in color calibration. So that it can give a proper ‘perceptual’ rendition of the colors .
Filmic doesn’t even come into play here .
Of you have the same problem, its easy to work around it be not using color calibration, and just using white balance module to set white balance.
Or take the time like I now do to get your white balance more in the ballpark in the white balance module , and then use color calibration to get it just right as I want it (and to fix gamut issues in scenes with strong colored led lights , or to fix lighting conditions with two different light sources with masking)
I didn’t say it’s filmic’s fault, just that I thought it was. Also, the purpose or intention of the colour calibration module was clear to me. But indeed, I overlooked that your issue arose with filmic disabled.
My question regarding the coefficients was related to the reported observations, colour calibration using Adobe coefficients rather than those in the exif data as legacy white balance does.
Those are just the r/g/b multipliers to get a starting white balance, right? How can there be ‘adobe’ coefficients? The raw file just lists that the balance as shot is something like 2.758 - 1.0 - 1.239. And that is what darktable-whitebalance uses for ‘as shot’, but also what Lightroom uses to get ‘as shot’ ?
The moment you use a white-balance picker, then whatever is in the EXIF data is not important anymore, the color picker data is used. The same if the raw software has an ‘auto white balance’ mode.
So I don’t get what ‘adobe coefficients’ are supposed to mean here? As far as I know, there is nothing special any kind of raw software does (or can do) here?
White balance coefficients are just a set of three multipliers (or 4, two for green… or something else for x-trans, but I know little of that), right? There is little magic that can be done in that.
Most of the ‘magic’ comes after that (demosaicing, curves, profiles, ‘color sciene’, tone mapping, etc…)
Hello,
Maybe I come a bit late, but here is what I could get with the presets I built - based upon Luc Viatour’s explanations - in scene-refered mode.
I did nothing else than import the picture, and voilà.
Some kind of “out-of-the-box” thing.
Maybe I could have do better, but I feel tired and a bit lazy too at this time.
Rgrds,
J.-Luc
The so-called “adobe coefficients” are the (usually) 3x3 matrix used to take camera data to XYZ. Definitely not the white balance multipliers.
I think that name got traction with dcraw, where the coefficients for each camera are stored in a humungous table in the program called “adobe_coeff”. Most of the datasets are extracts from Adobe profiles, hence the name. Here’s a link to a mirror copy, straight to the line number:
So to answer @aequalis, does ‘color calibration module’ use adobe coefficients… the module itself, no. Because the module itself does nothing with the 3x3 matrix / profiles.
color-calibration is placed in the pipeline after the camera input profile, and I think most of those are derived from stuff from Adobe? I don’t know, each camera has one profile in Darktable AFAIK, where they come from can differ.
I just tried to show what a noob can get with just a few settings in scene-refered mode. And I admit I did not get the courage to read all the posts. I roughly stopped at RawConvert’s one about an article he found at dpreview. It was turning too technical for my poor english knowledge.
As I wrote, I was tired, so I even forgot to use the tone equalizer module. Here is the xmp file with the correct settings. _DSC4665.ARW.xmp (7.0 KB)
Just one thing to end, I think picking the WB on the part you circled is a bad idea, if not a non-sense : this looks like a light wood part, neither white, nor grey, which only looks grey because of the bluring and maybe wrong colour balance.
Rgrds,
J.-Luc
Take a read through the “Mapping Camera Color Space to CIE XYZ Space” section of the Adobe DNG spec.
dt color science uses only the D65 ColorMatrix coefficients from the cameras.xml (on 3.9 branch; adobe_coeff.c if you’re on 3.8 and before), which are extracted from the converted DNG, and the WB multipliers later obviously. Convert your ARW to DNG and check the tungsten matrix - it might be significantly different enough to cause this (you can also play w/ changing these in cameras.xml on the 3.9 branch). Also check the AnalogBalance coeffs in your tungsten case (rawspeed/dt always assume those are identity, i.e. the camera doesn’t bake in any balancing in the analog domain before digitizing the raw values). Finally, the individual CameraCalibration is also assumed to be identity, so from rawspeed/dt perspective, XYZToCamera = CM_D65 (i.e. it is not interpolated away from D65, and not adjusted for analog balancing nor individual camera calibration).
Yes, different settings were tried in the color-calibration module. The illuminants change ‘the color to be balanced’ so they change all over the place. Comparisons were made with the picker in the exact same spot.
Changing the adaptation to XYZ seemed to help the most, but still way off from what the white balance module itself does when sampled in the same spot.
Again, it’s not about getting ‘a proper white balance for this picture’, it’s about the huge amount of difference in skin tones the color-calibration module seems to give when using the modern color workflow (and defaults) when sampled in the same spot vs just the white balance module sampled in the same spot.
Not using any kind of Sony camera profile seems to help a lot, and not using the ‘reference’ white balance but a more appropriate one in the white balance module (and then using color-calibration on top like in the modern color workflow) seems to help a lot.
This seems to indicate (to me) that the camera profile is making changes to the data that color-calibration then can’t properly work with.
Which is why I suggested to check the tungsten ColorMatrix1 from the DNG converter and try it instead as the input profile temporarily (as it will break existing edits)…
Quite a significant (>20%) change for the X (reddish) and Y (luminance/greenish) components, no? So that 1/AsShotNeutral value is a non-sensical starting point as the XYZ values from a “wrong” input profile could be quite different to what is assumed to be somewhat adapted already…
(Btw, including the CameraCalibration would make it even more precise, but not significantly as it is sort of close to identity, less than 5% for the blue camera pixel…)
And how would I go about using those numbers as an ‘input profile’ in Darktable?
And how do I recognize this ‘ColorMatrix1’ as being tungsten?
Or is it a file where the white balance is more towards tungsten (of even preset as ‘tungsten’) and then saved as DNG to see a change in ColorMatrix1 that Adobe uses?
Still gives me my question, how to ‘use’ that in Darktable as you say (it’s an interesting test for sure).
For what it’s worth: I found a custom-made Sony A7m2 DCP profile on the web (quite old, though). I used dcp2icc to get two ICC profiles out of it, one for 6500k and one for 2850k. I’ve both in the Darktable color\in folder, and they show up in the input-profile list, but I haven’t got good results with both of them.
Edit: @anon41087856: any idea white the picker did not work for color calibration?
As explained by @priort , this was due left-over values from trying colour matching once.
Seems okay for me…if I use the picker on the left part in the shadow or brighter I still get a neutral balance… picker values are two boxes one in the shadow and one in the brighter part…
Any chance you were doing some color matching…I find this a bit annoying…it is persistent with no indication…check and make sure you are on 50% luma and 0 and 0 for hue and chroma…may not be it but worth checking…
