When processing heavily underexposed shots made with an “ISO invariant” camera, I often encounter the double problem that
the sensor black levels determined by the camera are not the best possible ones,
while it is possible to find better black level values, all integer black level values introduce a slight color cast in the deep shadows.
That’s why I would like to experiment with fractional raw black levels. In fact, it seems to me that there is nothing physical that forces the black levels to be integers. They correspond to the intensity of some sort of noise (the sensor’s dark current or so), and noise intensity is a real (i.e. non-integer) value.
In a first step, I would simply like to experiment with fractional black levels to see whether they indeed help to address the color cast in the deep shadows. In a second step, I might want to try to add a color picker for black level determination, or perhaps even a full automatic mode.
Before I try working on this (I don’t know when I’ll have time…), I would appreciate comments on the following questions:
Has this been debated/tried before? (Obviously I searched, but could not find anything.)
Is anyone aware of existing algorithms for black level determination (I did not search yet myself)?
Looking at the source code, I do not see anything that forces the black levels to be integers, other than the parameters/sliders being defined as integers. In fact, the actual normalization computation works with floating point. So I don’t see any principal problem with switching them to floating point.
Would it be feasible to switch the black level parameters to floating point, while keeping backwards compatibility with existing edits? (Any pointers to a description how backwards compatibility works in Darktable are appreciated.)
They are. Another reason not to go to fractional values too soon…
It would be good to know if the cast is caused by a “wrong” black level, and if so, why.
A fractional black level could be useful, if the cast is caused by a wrong black level, and the cast changes hue or intensity with the black level
The raw sensor data as well as the black levels provided by the camera in Exif are indeed integers. But this does not mean that the real black levels are integers. An average of integers is typically not an integer.
Let’s imagine a monochrome sensor that should measure perfect black, but due to dark current it measures (for some series of pixels): 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 1, …
(I know that in a real imaging sensor the black levels are more like 256, but the above is easier to read.)
So let’s say that we have determined that in the case of our sensor for a perfectly black frame a zero will be measured with probability 2/3, and a one with probability 1/3. What is the appropriate black level? In fact, it’s neither 0 nor 1 but 0.333.
The situation with camera sensors is exactly the same. There is nothing fundamental that forces black levels to be integers. The Exif tags were defined as integers, because this was sufficient for cameras that were not ISO-invariant. There was so much noise that it didn’t matter anyway. But for modern sensors it matters if the shot was heavily underexposed and is pushed in post. One integer-valued black level can be too low, while the next one already too high.
Actually, i don’t have a clue what you are trying to achieve. If the sensor provides integer data, those are integers and you convert to float by calculation using white point and blackpoint. Blackpoints are per RGB channel (photosite) for color raws and just one level for true gray scale images. There is no averaging involved anywhere. There are a few images/cameras where the firmware gives bad data but those are (very) rare, some sony cameras where affected iirc.
About the color cast, if you in/decrease the blacklevel for a RGB photosite that of course changes the resulting float (or fraction) so white balance goes wrong.
We can consider a grayscale sensor for simplicity. Thus there is a single black level for some given shot. If we want to measure that black level, we can photograph a dark frame with the same settings (shutter, ISO) and under the same conditions (e.g. sensor temperature), and we will obtain a full frame of pixel values that are somewhat noisy. That noise is not photon shot noise, since the “scene” is perfectly dark. It’s the dark current of the sensor and the read noise of the electronics.
On average, this dark frame should be mapped to black. Therefore, the most appropriate black level is the average of all the values in the dark frame (excluding any hot pixels etc.). The precise value is not an integer. This is the average that I mean.
Already today (if I understand the code correctly) rawprepare.c will not saturate at zero when subtracting the black level from the pixel values:
out[pout] = (in[pin] - d->sub[id]) / d->div[id];
Thus a value that is just under the black level will be mapped to a negative number. This is (I believe) the correct thing to do, otherwise a bias would be introduced. So it seems perfectly possible to subtract a fraction instead of an integer. (In fact, the type of d->sub[id] is float already today.)
All of the above only matters for heavily underexposed shots that are pushed in post-processing. I will try to prepare a demonstration.