Gosh! I tried that once, and didn’t realise the setting was persistent, not even removed by resetting the module or applying presets. Thanks! And thanks for opening
1 Like
The issue k had btw wasn’t in the whites or greys. It are the reds that shift quite different , so I wonder if a greycard shot would reveal much.
I develop my original shot like this (as a test ):
- Reset everything to default / discard history , disable filmic and disable highlight reconstruction
- Set white balance to reference
- In color calibration take a sample on the lower left corner
- Put a lab color picker on the cheek of the baby, and raise exposure in the exposure module until the L value says +/- 55
- Sample color calibration again just to be sure
- Enable denoise profiled with auto / default settings
- Enable filmic , set the ‘extend dynamic range’ slider or whatever the name is to 10% (right click to enter value ), then hit the auto button to set black and white. The rest is at the filmic V6 (DT 3.9 ) defaults.
- Enable color balance RGB with the 'more colorful ’ preset
- Enable local contrast with default settings
Now, take a snapshot. Then discard the history , and begin all over again , but now use the legacy color method (as in , disable color calibration and use white balance to sample to lower left corner ).
Finally compare with the snapshot taken earlier. There are now (almost) no luminance differences , since the exposure is matched to L 55 on the cheek.
The white of the bed is basically spot on the same. The grey in the lower right has slight differences but not that much. The blue/green/teal of the blanket is different but not even that much (not so much that I would be bothered by it , personally ). But the skin, just the skin , is the difference between warm baby skin tone and just plain 'make fun of Sony colors ’ magenta / purple .
It’s not that ‘there is a bit of magenta’ to the skin tone, no it really turned magenta , like an alien :).
Bit this is somethingi just seem to have to live with , that the Sony color profile is quite different depending on the scene lighting and I haven’t found a profile yet that works and doesn’t have this (except not using an input profile at all). Setting the white balance to be more neutral before going into the rest of the pipeline seems to be my trick now , but I wonder if color calibration is then still doing something or if it’s just a noop now (or a glorified gamut compressor )
Hi there! I believe color calibration might just be working as it should. Let me explain:
The WB module is set to camera reference. Then the input color profile does its thing (it is known to be bad for sony cameras), then comes the color calibration module and with the picker you white balance the image, meaning that there is no color cast in the image, so whites are white. But your input color profile is still wrong, which means your colors are wrong, even though white is white. To make things worse, the picture is shot under artificial lighting, which normally has a bad CRI.
What I believe you should do to fix your colors would be to shoot a color checker under that light, and create a color calibration profile for your camera under that light. If you want a more general purpose profile, you should do it outside under natural light. You can save those as presets and have them automatically applied to your images. This changes the coefficients in the R, G, and B tabs, correcting the input color profile. Aurelien has a video explaining how to do it. You might also want to create a custom White balance profile for your camera.
This is the video on color calibration profiles:
This is the one on how the module works:
A general profile for daylight is not going to fix this, it will yield the same issue as the input profile has now :). (My) Sony camera just needs a lot of correcting. So when the white balance is way too warm , some things get overcorrected. Then color calibration tried to work drom that , and goes in the wrong direction.
So if i make a camera profile myself under daylight , that will have the exact same quirks as the default one I’m guessing. Other a7m2 dcp profiles online have the same behaviour .
Making a profile under more tungsten conditions would help though , but then I need to make a profile under tungsten conditions but with the white balance still set to reference (which is basically daylight for me ).
Either way I need to or) select a different white balance as starting point depending on the picture or) select a different a different profile depending on the picture .
They are both a hassle if you are working through a whole bunch of files .
Just be careful to check when doing your wb testing that under the color matching tab you have 50:0:0… Currently the setting are persistent and even stay between DT sessions so if you have ever used it and not manually reset it any spot an corrections will be wrong…I am not saying that you have the but that is what happened to Kofa
The suggestion above creates a set of channelmixer adjustments to compensate for your profile and make the colors accurate and so you may be able to come up with essentially what are a set of tweaks to correct for the input profile. So a daylight correction and tungsten etc…Any camera will benefit from this. Some times the change will be minimal and other times significant…
1 Like
Are you talking about a color calibration profile, or setting your illuminant to daylight? The second will not fix your issue. The first very likely will.
In the current scene referred workflow, the white balance module should be set to camera reference, unless you have created a specific white balance profile with a calibrated monitor. As Aurelien has already mentioned in his videos, color calibration is meant as a corrective tool, not as a creative one. When white balancing, it should bring your picture to a neutral state (neutral whites, grays and blacks). If you make the white balance too warm, it will only further mess up your colors, and that info gets passed onto the other modules in the pipeline, like color balance rgb. If you want to make the picture “warmer” or “colder”, color balance rgb is the better place to do that.
Not necessarily. The default one seems broken for your camera, and I can’t speak for the a7m2 dcp one. For my camera, a custom color calibration profile has a significant impact.
If you shoot under those conditions a lot (indoors and with home lights), a custom profile for that light would give you the best results, although a general purpose one will likely already improve things for you.
I am not sure that I understand you correctly. The current workflow is the following: don’t touch the white balance module, just leave it on camera reference. Use color calibration to get a correct white balance. If the input profile is bad, like in your case, create a new one with a color checker as I explained above and save it as a preset. This can be automatically applied to all the images from the same camera. Once you do that, the only thing left for you to do when adding new pictures is to use the color picker in the CAT tab to select a neutral spot (white, gray) and that will remove any color casts. Once you set things up, it is as quick, or perhaps quicker than most other raw processing programs.
I would recommend that you watch those videos by Aurelien, and take some notes to fully grasp what is in there. That’s what I did.
Thanks for the help - really! - but did you read me first post?
The problem is not ‘accurate’ color (I don’t often care that much) or a correct white balance.
The problem is that if the scene is shot under more tungsten-like lighting, that the combination of ‘camera reference white balance - input profile standard matrix - color calibration pick a neutral patch’ just plain goes wonky and wrong in certain hues.
While the same thing for pictures in more-or-less daylight situations are just fine!
I can ‘fix’ it (no, work around it) by not using the reference white balance in the white balance module, but a ‘better one for the lighting conditions’.
Disabling the input profile - as in, setting it to linear rec2020 while also using the default linear rec2020 working profile - also ‘fixes things’, but then you have no input camera profile at all.
So the problem is that ‘using a single camera reference white balance’ is just not going to work for my camera, as long as the ‘standard matrix’ input profile is in use.
So the problem is that the result of color calibration on a neutral patch is really different on just some of my pictures.
If I fix that by using ‘a different reference white balance’, or by ‘using a different input profile’ or by ‘using a color-calibration preset that has R/G/B values set to correct for things’ are all possible methods, but they don’t fix the fact that I first need to ‘discover’ that I need them, because the problem is not there on all pictures… which makes editing a hassle. Because you first have to discover that “it is a problem picture”, and then enable a workaround for problem-pictures.
I know the idea is to always use ‘camera reference white balance’ and then not touch the module, but the fact is that it’s not going to work for my camera if that reference-white-balance is too far off from the correct white balance for the scene the picture was shot in. If the reference white balance is too far off from the reality of the scene of the shot, the input profile ‘standard matrix’ will apparently mangle the image so that color calibration can’t get normal colors anymore.
Take a try on the picture in my opening post. Use the legacy color workflow (just white balance, no color calibration) and neutralize on the lower-left of the picture.
Now start over, set the white balance to camera reference and try to get somewhat close with the colors with using color calibration, neutralizing on the same spot. You can’t (well, maybe you can by using the R/G/B tabs in color-calibration, but then you get my final issue, that those values are only to be applied in some cases, not all).
I don’t have a color checker, so I can’t shoot the pictures to try getting it right myself. But I also think that it can’t be done with this camera. It appears, a profile that is done in daylight conditions will always apply certain corrections that should not be done in tungsten conditions.
@kmilos had a nice test, because Adobe apparently writes quite different ColorMatrix values to a DNG file with his camera, depending on daylight or tungsten situation. So - at least as an experiment - try to replace the ‘standard matrix’ in input profile with something of the ColorMatrix that Adobe writes in tungsten situations might be interesting, but I have no idea how to do that.
I think the real advantage of CC is that it is maskable and so you can attempt to deal with multiple difficult lighting and multiple lighting …in your case given as you say it can be a pop up issue with global WB…using legacy WB is not an issue I don’t think so go for it…
One question I would have that your situation brings up…is are the legacy WB D65 values only important if you want to have the CC module produce the as shot WB accurately…If you are going
to do a spot WB with it does it matter what you start with as the calculation will be base on an are in the image?? I might need to think a bit on that but where I was heading is lets say you use CC with D65 WB and now you use a second instance…it will use the data from the previous instance not from the WB module any longer I would think and so that part of the image is corrected based on the output from the first CC module?? Again I could have this part all messed up I am just thinking off the top of my head…
So, your camera’s “reference white balance” struggles under tungsten. Make a camera profile AND white balance reference under that illumination. And then, use them both for all sequences of images under that illumination. PITA, but that’s how you handle it.
I’ve made camera profiles from target shots that also correct white balance - not hard to do with the right tools (cough-cough, dcamprof)
This is the point behind the dual-illuminant DCP camera profiles: two color matrices, one daylight, one tungsten, and the raw processor takes the illuminant selected by you and interpolates an appropriate matrix for that illuminant. AFAIK darktable doesn’t handle DCPs that way… RT does, IIRC.
In any event, you need to work on what it takes (profile, wb multipliers) to make a linear image of a target look like that target, in hue. Then, you have the correct reference for departure in both tone and color with whatever tools you choose.
1 Like
From what I gather and seem to discover , color calibration doesn’t ‘use’ anything from white balance module.
I can set white balance waaay too cool and then do a ‘as shot’ and CC will make the picture look as shot… I can set white balance waaay too warm and then reset CC to ‘as shot’ and it will set itself nicely to whatever was ‘as shot’, even if the input white balance is completely different.
White balance in reference was only needed to get something workable for demosaicing . Not because CC needs it.
Because input profile comes after white balance, your chosen white balance can affect what input profile does . This is according to the docs particularly true for Sony / Olympus camera’s .
Since CC comes after that , it all might affect CC. But it’s not like CC uses info from it.
Multiple CC instances are ok because of all this. But the CAM adaptation is better suited to be used once IIRC , so that’s why new instances of CC start with adaptation set to none .
@ggbutcher i realize that making a complete pipeline calibration for a specific lighting condition is the best way if you want to go all out color-nerd.
But I have no color target, and k don’t want to go down that route. I also don’t demand ‘perfect’ color or require a certain artistic approach. I’m more often very happy if it looks ‘kinda right’.
But a tungsten shot, WB’d as daylight from my Sony camera will just never look ‘kinda right’. Since it’s a selective hue range that goes wrong , the whole image can’t be CC’d.
I haven’t tested the color-matching, what the picture will look like if i force the skin color to be matched , If everything else will go wrong 
Here’s a color checker image shot with your camera:
Create a profile with it and try. Lighting will be different, but perhaps it improves things a bit? It should fix your broken input profile somewhat.
I don’t know what your camera does, but it seems like the bad colors come from bad lighting and a bad input profile, that then get further twisted by CAT. See if this improves things.
I tried with an image from Imaging Resource (not that one, but another test scene with a color chart in it shot at base ISO at f8). I believe their lighting is closer to D50 and not D65, but I don’t know if that matters much.
Well, to be honest. I tried the thing aurelienpierre shows in the video: To use the color-calibration colorchecker mode to set the R/G/B tabs. Not to create a real profile from it.
… but, that doesn’t help at all. Once again, the problem is not getting more accurate colors, the problem is the built-in input profile Darktable uses for my camera, which destroys things for color-balance if the white balance is too far off.
I can try again if I manage to get a real profile from it.
But I think the fact is still this: The input profile adds a lot of red, apparently my camera/sensor needs that compensation in daylight.
But if I shoot an image where the camera picks +/- 2800k as white balance, but you set the white-balance module to reference (+/-6500k) the image is very orange. The input profile then adds even more red (which affects the entire image).
Then, CC tries to neutralize the image and messes up the reds now (since the input profile messed them up before).
Starting the whole process with the white-balance module set more to reality (+/- 2800k), and the image is way more neutral going into the input-profile module, which then does NOT messes up the colors but actually does what it is supposed to. Then CC comes on top and works fine (and can now even benefit from more accuracy by use of a R/G/B calibration).
Ok, I fetched the image you linked (well, the raw version of it).
Followed the dcamprof way to go from (rawtherapee save as reference image → argyii scanin → dcamprof make-profile → dcamprof make-icc) to an input profile.
Then, used the same picture to do the ‘white balance preset’ way: Disable everything, get a white balance picker on one of the grey patches on the colorchecker to get my ‘reference white balance’.
Then did the ‘use color-calibration to get even more accurate colors’ thing, so using the colorchecker-profiler baked into color-calibration module to set R/G/B values. This is with my new ‘reference white balance’ preset set, and using my own ICC created my dcamprof.
Got a profile rating of ‘very good’.
Then, tried using it to edit the picture linked in my opening post again.
Still the same purple thing (although I get the feeling it’s a bit less now?).
Then redid the edit, but with white balance not set to the reference-preset but to a more neutral ‘suitable’ one by sampling the lower-left, still using my own custom ICC, then using the custom calibrated color-calibration preset created earlier, and then sampling the same lower-left part in CC (ie, what I did before to workaround the problem → not use a reference white balance in the white balance module).
The difference is still very stark.
The idea of a single reference white balance is just not possible with this camera…
I redid the edits with the Darktable built-in camera profile, and the normal reference white-balance vs neutral white-balance. The difference is clearly much worse, so going through the whole calibration pipeline helped a bit, but didn’t (really) fix it:
_DSC4665 - test neutralwb.ARW.xmp (18.5 KB)
_DSC4665 - test referencewb.ARW.xmp (12.9 KB)
I had an other 
moment.
The problem was in the input profile . Not using the standard matrix but using ‘linear rec2020’ as input profile doesn’t even look that bad, and makes CC work ok no matter the input white balance and lighting conditions.
What if i go through the color-checker-in-CC routine to fix colors somewhat, when using linear rec2020 as input ?
So i don’t use any camera profile at all, but use the RGB sliders in the CC module to get them more in line.
The results don’t differ that much at all from what I can see compared to using a color-checker in CC together with the standard matrix (but not using reference white balance ).
So i might just switch my defaults for my Sony to always have my measured WB preset , linear rec2020 as input profile , and then fix white balance in CC like 'you are supposed to ’ and just add another instance of CC with the calibration preset on top to fix colors some more.
Will have to do some tests if the colors always come out ok enough though .
I would suggest to calibrate the wb module by using the colorpicker on a picture of a D65 calibrated screen.
Then keep the generic matrix color profile and try loading the camera wb in Color calibration module if the calera was on auto wb, and undo the dominant color by hand if there is one.
If the skin is not ok but the rest is good, then it’s more likely a spectral issue and you will have to correct the hue locally.
That’s the issue explained in the opening post. The input profile wrecks color-calibration so that a single hue-range is thrown off, only if a) using the standard color matrix in Darktable and b) if using a white-balance that’s too far off (like the reference white balance set while the lighting conditions were clearly different than daylight).
‘calibrate the wb module’ will not fix it, using the standard profile/matrix will just cause the issues.
If the skin is not ok but the rest is good
Exactly that, but the problem isn’t there when the white-balance module is used to pick a better suitable starting point, or no input-profile at all is used. The lighting situation wasn’t ‘weird’, it was just tungsten. There is no spectral issue in the shot, the white-balance module handles it perfectly. It’s just CC that can’t handle the situation if the white-balance is set near daylight.
@anon41087856 This is a different issue I’m guessing from what I’m reading in the color-calibration docs for Olympus/Sony cameras. There it’s just said that the CCT is detected as ‘invalid’ which will only affect what temperature is displayed in the CC module, not the effect of the module.
In the picture from 1st post:
- WB: reference, input profile: standard, CC: picker used on lower-left-bottom: Good whites, purple skin (wrong)
- WB: reference, input profile: linear rec2020, CC: picker used on lower-left-bottom: Good whites, good skin (good) (colors slightly off because of no profile used)
- WB: As shot, input profile: standard, CC: picker used on lower-left-bottom: Good whites, good skin (good)
- WB: Picker used on lower-left-bottom, input profile: standard, CC: disabled: Good whites, good skin (good)
- WB: Picker used on lower-left-bottom, input profile: standard, CC: picker also used on lower-left-bottom: Good whites, good skin (good)
If using a testchart shot made by someone else (unknown lighting conditions, D50 / D65?)
- WB: Calibrated on grey, input profile: standard, CC: picker used on lower-left-bottom: Good whites, purple skin (wrong)
- WB: Calibrated on grey, input profile: ICC made with dcamprof from the same testchart shot, CC: picker used on lower-left-bottom: Good whites, purple skin (wrong)
- WB: As shot, input profile: ICC made with dcamprof from the same testchart shot, CC: picker used on lower-left-bottom: Good whites, good skin (good)
It seems as if any combination of ‘standard input profile’ and ‘white balance too far off’ just throws CC off.
@anon41087856 Is this by design, or just a limitation of the code? Or is it a bug? If you give CC an image which is waaaay to warm, should it still neutralize everything OK, or did the Sony standard profile already destroy it then?
The same I guess if you give an image which is waay to blue / cold (like an underwater shot but with the white-balance still set to reference / daylight). Is it expected that the input profile + CC still work fine, or is it expected that some hue ranges may be completely shifted because the input profile was applied to an image with (very) wrong white balance?
Is this more what you were trying to achieve? I don’t know how close the colors of of the blanket, face or hat are, since I didn’t shoot the picture, but you can mess around with the module to your heart’s content, once you understand what it does.
As someone mentioned above, you can mask the color calibration module, so you can work on the face specifically without affecting the rest of the image.
I reset the raw black/white point module, since it was way off. I then adjusted exposure and filmic. In this case I used a first instance of color calibration profiled from the picture on imaging resource (not sure how helpful it is, since the illuminant is way different in your picture vs the color checker one). Then I created a second instance of color calibration, created a parametric mask for the reds, and tweaked them via the color mixer. Once you are happy with your results, you can create two presets, and apply them to difficult images like this. Finally I added color balance rgb with the add basic colorfulness preset.
Keep in mind that these are far from ideal conditions. You are shooting at ISO 6400 under poor lighting, with a camera that has a bad input profile. This is the embedded JPEG preview from your image:
But masking isn’t needed at all if you use pure legacy color workflow or if you use the as shot white balance as a starting point for CC. IT isn’t a hard picture to neutralize , there is a single light source and no different casts .
A single picker of the white balance module does what you have done with masking and multiple instances…
This seems like a problem with the workflow , not a problem with the shot.
The whole idea of ‘one reference’ white balance is to give the demosaicing something to work with, right? So what’s the harm in picking something else there and still using CC?
And if a camera profile tries to fix color response , doesn’t it need a proper white balanced image to start with ? In my head , you can’t give a completely wrong starting point to an input profile and expect it to still work as intended , right ? And CC comes after that, so of course it goes wrong because it receives faulty input from the camera profile .
White balance needs to be before demosaic , exposure needs to be before input profile , and I’m thinking CC (at least the first instance) would also have to be before the input profile . After exposure (because the CAM needs to know the correct lightness/brightness/exposure to make an attempt at 'perceptual colors ') but the input profile is like a calibration / color fixer… doesn’t that need to go after we have a good proper starting point ??
I think there is a reason the modern color workflow isn’t the default yet in Darktable… From what I encounter here the current module order seems to only work ok if your camera has a very linear / consistent color matrix, and that’s not all of them.
Who knows other camera’s have this as well ? @ggbutcher took a test shot of a grey card, but there is no skin in there picture so we can’t see if it has the same problem.
Maybe you can retake it but with a finger or a hand in there as well ? Shot under 'bad indoor lighting ’ but with a grey card so we can neutralize it ok, and then see if the skin tone comes out somewhat believable.
Can you share your version with the legacy WB? Just for curiosity? If you still get better results from the legacy WB, it is likely because you don’t have a custom WB profile for your camera, so D65 is not really D65.
The problem is with the lighting in the scene. Home lights don’t reproduce colors right, they have a bad CRI. Color calibration is an amazing tool if you know how to use it. Nobody is stopping you from using the legacy WB if you get the results you want, though.
I’ve posted multiple versions, with xmp files. A few posts up even:
The last two in that post are with stock Darktable profiles.