Also see this thread Measure black and white levels?
För Canon users, especially Measure black and white levels? - #11 by Peter
I don’t think black points are directly related to noise though — if that was the case, we would see non-uniform values from a “black” scene (eg with the lens cap on), and would not be able to simply subtract it.
My understanding is that black levels are simply an artifact that comes from a reference voltage used by the ADC.
Yes, but I think that is incorrect.
But I was just being pedantic. I don’t understand why
If Darktable knows the camera, it should have the correct value. Otherwise it needs to be added. The user should never have to fiddle with either the raw black or white point.
I think in total that is essentially it but it can be broken down from what i read in the links on the noise generated by the various resting currents… which likely fluctuate with temperature variation.
THere is something called a bias frame that can be used with the dark frame to partition these sources a bit…and some of the noise is fixed but some can be random though like so small as not not need this sort of breakdown and correction… a threshold subraction will generally be enough I guess.
The sensor itself with the photosites and then the subsequent electronics processing that data each have a contribution… again I am not expert and I was just relying on the information in the wiki links to black noise and readout noise… included in the blog. It sounds reasonable to me…
I think that
\begin{multline}
E[\text{raw pixel value}_i] = \min(\max(\text{black point}_i, \\\text{bias}_i + \text{exposure time} \cdot \text{dark current}_i +\\ \text{actual freakin' photons}_i), \text{white_point}_i )
\end{multline}
where white point is usually independent of a photosite i and black point just depends on the channel, mostly.
Black point is affected by the noise in the sense that the camera manufacturer takes it into account when determining the reference voltage of the ADC, but for the user it is just a given for a particular camera and sensor.
Yep there is the physics of it, the type of sensor CCD vs CMOS and the quality/stability of the components, design of the circuitry etc …when combined and evaluated by performance testing that information gets used to inform the manufacturer what value to set for the threshold.
For the most part the user only needs to know that number and know or hope that it is a good well grounded reference… Engineers on the other hand might want to know if they have a great sensor but a flawed amplification/processing circuitry or thermal control etc to make a better design choices and/or get the optimum performance specs…
In rawproc, I have a blackwhitepoint tool. It’s an explicit part of the tool chain, among its different modes is “data”, where the black and white points are set from the minimum and maximum values in the particular image. Now, I know this probably wouldn’t play well with “low-key”, or dark, brooding images, but for the majority of what I do it works just fine, also with any other raw I download for PlayRaw.
Oh, to confuse things, my subtract tool has a “camera” mode, where it’ll read whatever black level is provided in the metadata and apply it to set a black point. It’s what I use by default for raw images, so the blackwhitepoint:data tool I put in later in the toolchain doesn’t really do anything with black…
All this just to say I’d rather manage this operation explicitly, rather than relying on a database of settings. Yeah, I have eventually come to some defaults, but I did so with at least a cursory understanding of the implications. Oh, and thanks, @rvietor, your previous post lays out a nice perspective on the situation…
YMMV, all…
I’ve always been puzzled by Canon …
… are their sensors so bad that they have to lop over 12% off the histogram LHS, so to speak?!
Y’might ask the question at the PS&T forum at DPReview, get someone like Eric Fossun to answer.
Also, pull some black level metadata from downloaded PlayRaw raws, see if your notion correlates. I’d do it, but I’m on my tablet for the next few days and I deleted most of my PlayRaw downloads on it before I left home…
I am not sure you got what you asked for, so maybe a more detailed description would help.
The photosite in your camera corresponding to each pixel converts light to electrons. This charge is then converted to an integer using an analog-digital converter (ADC).
ADCs lose precision around zero, and thus have a minimum voltage. Extra voltage is applied before charge is converted to an integer, so the “black level” corresponds to 0. They also have a reference voltage, the maximum they are designed for, above which they clip.
While very sophisticated ADCs exist, camera manufacturers have to stuff the whole thing into a tiny package that is not very good at dissipating heat and make it operate on a small battery. Also they have to think about the amount of data that flows through the pipeline.
Because of these constraints, they do not design the ADC for more precision than needed. The theoretical maximum would be the well capacity of the photosite, which ranges from 2^{14} to 2^{17} in current cameras (depending on vintage, sensor size, and pixel count), improvements are happening but they are small and incremental, but capturing all of this would just blow up the raw file for no good reason and slow down exposure too.
So they target the number of bits b, eg 12 or 14 (in recent cameras, or even more in medium format), designate an integer that may be below 2^b as the white point (because ADCs have noise, pixels can be counted above this, so they leave room or have to clip it in the pipeline), and pick some positive integers as the black level (corresponds to the minimum voltage). Black levels are getting lower and lower with more recent sensors.
A black level that is above 10% of the white point may look like “dynamic range” was sacrificed, but in practice the bit depth already overshoots what the sensor is capable of so there is no practical loss.
TL;DR:
- black and white points were determined in accordance with the sensor and ADC design,
- the raw conversion pipeline has to know about them, but they are not something a normal user would be concerned about,
- high black levels do not mean that dynamic range is sacrificed, it is just an offset voltage required by the design characteristics of the ADC.
3 Likes