How does white balancing work in darktable, and why are there two modules involved? What is the difference between the ‘legacy’ (white balance only) and ‘modern’ (white balance + color calibration) methods?
white balance performs a simple per-channel adjustment on raw sensor data. When it sits before demosaic, colour does not exist yet, all you have is per-channel sensor readings. It could have been placed after demosaic, but that (and I think also highlight reconstruction) actually work better if neutrals are correct. For example, on severely blown areas, you may discard colour info, and just go with luminance, producing shades of grey or even white, but if subsequent white balance operations multiplied channels with different values, the reconstructed highlights would become tinted.
So, what’s wrong with the simple method? By applying multipliers to channel values, it creates an ‘imaginary sensor’, where, for example, the red channel is 10% more sensitive than in reality, while the blue channel is 10% less sensitive. This is not the sensor that the camera profile describes. Looking at it from the other point of view, ‘increasing red’ in the input by 10% and ‘decreasing blue’ by 10%, we pretend that the sensor’s pixel groups (the red, green and blue cells) are only sensitive to some kind of ‘red’, ‘green’ and ‘blue’, so, for example, adding more ‘red’ does not influence the ‘green’ cells. In reality, the sensor’s cells (and the cells in our eyes) have overlapping sensitivity ranges.
So, the ‘modern’ method tries to improve on that.
First, white balance is used, either in ‘camera reference’ or in ‘as shot to reference’ mode. There’s nothing special about them:
- ‘camera reference’ uses white balance multipliers derived from the camera’s characterisation matrix. The derivation is as follows: for each camera, either from the raw library’s database, or embedded in a DNG file (dt only uses 1 matrix, even if the DNG embeds 2), we have a camera characterisation matrix, which provides the mapping between XYZ and sensor RGB values. Unlike most XYZ → RGB matrices, this one does not map the reference white point to R=G=B=1, but directly to the sensor readings, e.g. to (0.5, 1, 0.67). The reciprocals of those values are exactly the ‘camera reference multipliers’, because applying them to camera readings of RGB=(0.5, 1, 0.67), the result is (R=G=B=1)=white. These reciprocals are the values you’ll see in white balance, when it is set to camera reference mode. The ‘white balanced’ result is not perfect, but also not horrible. For example, if you shot a warmly lit indoor scene, it will appear very warm, but still better than the all-green we would get without applying any white balance at all.
- ‘as shot to reference’ does two things:
- it applies the camera’s as-shot multipliers, recorded in the EXIF. The hope is that these are even closer to ideal than the camera reference multipliers, so demosaic and highlight recovery perform better.
- it also sets a flag called ‘late correction’.
After demosaic and highlight reconstruction, image data reaches input color profile. That uses a matrix derived from the characterisation matrix, which takes the white balanced input (white is R=G=B=1), and maps it to D50-referred XYZ values using chromatic adaptation. From there, we are ready to go into the D50-based pipe working space.
If using the legacy method, white balance is responsible for mapping sensor values, using the RGB multipliers, so that a neutral spot is placed at R=G=B. From then on, we pretend the scene was shot under the matrix’s reference illuminant, and input color profile applies the camera profile, and remaps everything to the D50-based working space.
If using the modern ‘camera reference’ method, the incoming data is camera-referred (we used the multipliers valid for the reference illuminant). The image may not be properly white balanced (it may look too warm, cold, or have a tint), but it looks as if we looked at the scene with eyes that got used to an environment illuminated by the reference illuminant, and suddenly peeked into a room showing the scene, under its own illuminant, before our eyes could adapt to the light in that room. But that’s OK, as color calibration is the one to take care of that.
Finally, if using the ‘as shot to reference’ method, input color profile will see the ‘late correction’ flag. By multiplying by (camera reference multipliers / EXIF multipliers), it will undo the effect of the EXIF multipliers (which still served well to improve demosaic and highlight reconstruction), and return to the state balanced for the camera-reference, then the same processing is done as for ‘camera reference’. Then, the data goes to color calibration.
So, when the data arrives to color calibration, ideally it is the D50-referred data, converted in input color profile from the original raw data using reference multipliers (either right from the start (‘camera reference’), or via the ‘EXIF → reference’ path (‘as shot to reference’, ‘late correction’)).
If set to as-shot mode, color calibration will then read the EXIF multipliers (otherwise, it will use the values set via the GUI or selected with the picker), and do the maths to figure out the xy chromaticity coordinates of the original scene illuminant. For example, the camera’s reference multipliers could be 2, 1, 1.5 (so illuminating a grey card with that reference light, and then multiplying R by 2, blue by 1.5 would result in R=G=B). However, you shot a scene under warm light, and it’s enough to multiply R by 1.3 but you must multiply B by 2.8 to get R=G=B (WB coefficients of (1.3, 1, 2.8)). How can you get the illuminant from WB multipliers?
Pushing the reference white point coordinates through the XYZ → RGB (CAM) characterisation matrix provides the raw sensor values that represent a neutral spot under that reference illuminant. It can also take any XYZ value to the camera’s sensor readings. Its inverse works in the opposite direction, takes (CAM) RGB to XYZ.
We saw that for the reference illuminant, the sensor values are the reciprocals of the WB multipliers. The same is true for any set of WB multipliers. If the camera needed to use multipliers (1.3, 1, 2.8) under our warm light to get white/neutral, the sensor readings were 1/1.3 = 0.77, 1, 1/2.8 = 0.36. We can run those RGB numbers through the RGB → XYZ inverse matrix, and get the XYZ coordinates of the actual scene illuminant. There are a few more steps because of the camera reference illuminant → D50 conversion in input color profile, but I’ll skip the details here.
Once we have the XYZ (and therefore xy) coordinates of the original illuminant, we can construct an adaptation matrix. This matrix is no longer a simple per-channel RGB multiplication like we had in white balance: it allows ‘crosstalk’ between channels. The result is that it can take pixels from the scene illuminant to D50, and place colours where they should be.
- I was talking about RGB ↔ XYZ for the camera matrix and its inverse, but it also works with 4 channels (the code uses a so-called pseudo inverse, which can be calculated even if the matrix is not a square matrix).
- The camera reference illuminant is (usually?) D65, I think, and D65 is hard-coded in a few places in the workflow.
I have a PR (not coming in 5.6) that would allow one to use manually selected (e.g. picked) WB multipliers in white balance along with color calibration (so have as neutral whites as possible for demosaic etc., and still perform proper chromatic adaptation). In reality, the difference is hard to notice, the feature may not come at all in the end.
Since color calibration is after input color profile (in the D50-referred part of the pipe), the CCT (‘temperature’) values that is displays are not the same as what you’ll see under white balance, where everything is relative to D65. Don’t worry about those numbers too much. If you want something close to what other editors show, check the number from white balance.
As for IR: since the matrix no longer describes the sensor, you won’t get ‘accurate’ colours. But that’s not the point of shooting IR, anyway. 
(Now people who actually understand this stuff can come and correct me. This is my understanding based on the code, so I’m pretty sure some of this is very non-scientific and might even be wrong, not just sloppy.)