Quick math questions

So it seems you are assuming that the domain of the log2() function coming from an input device is restricted to [1,256]. This is a wrong assumption:

  • The values from a camera are generally 12 or 14-bit, and in a raw editor they are generally mapped onto a floating point number in interval (0.0, 1.0].
  • The values direct from camera need to be white-balanced, which means some values can be multiplied, which can result in values >1.0.
  • There may be an exposure correction, which can end up scaling the values up even further. So in a true linear scene-referred pipeline, the individual channels will be floating point numbers >0.0, but the maximum value depends on the processing steps that have happened in the meantime.
  • At some point you are going to want to transform into another color space (eg. Linear Rec.2020), which will result in the numbers shifting around, possibly even resulting in negative values if some of the colours in the original image are out-of-gamut in the new colour space.

So, the input to the log2 function will be a floating point number, generally on the interval (0.0, 1.0) but not necessarily, and of course for 0.0 < x < 1.0, then log2(x) will be negative, and for x > 1.0, then log2(x) will be positive. The EV values are relative to when x=1.0, but that reference point is completely artibratry in a linear scene-referred pipeline, and so EV=0 doesn’t hold any special meaning.

My thoughts were in disarray. I think it is clearer now. So

One

[       -12,0]=log2(1/[1<<12,1]) # 12 bits? Perhaps, but consider it more as
[0.00024414,1]=     1/[1<<12,1]  # where ≤0.00024414 is essentially the epsilon

I could refer to the suggestions made above to deal with or avoid spurious values.

Two

The range doesn’t matter as long as I treat values the same way and adhere to a reference point. The lower bound could be clipped to deal with one. I am unsure about >1 but I suppose that it would involve remapping or compression to bring the image to an 8 bit encoded image.

Outside of colour maths, what I usually do (in engineering computing) if I know that R is not negative, is to compute M = log( (G+eps) / (R+eps) ) – where eps is the machine epsilon (i.e. the smallest number it can deal with that is not zero).
An alternative would be to use M = log( G / (R+eps) + eps ) – but that would not result in 1 if both G and R were zero.

The above has the advantage of working also in “blind” cases, i.e. a spreadsheet, matrix operations (where G and R are large arrays with lots of values) or other environments where you might not want to include any conditionals, but the disadvantage of introducing an error (the tiniest computationally possible error!) to the result.

Although that’s a tiny error, you might be particularly worried about continuity in some cases, but not that worried about execution time. In this case, I’d firstcheck if either B or G is zero, then:

  • if both are zero: M = 1
  • elif R is zero: return largest positive value which can be represented with the variable type that’s being used
  • elif G is zero: return largest negative value
  • elif neither is zero: return log(G/R)

This means that if you drew a graph of M as a function of either G or R, it would be nicely continuous but “clip” at the edge of the range that can be represented. In particular, if you plotted a graph where G = R = f(t), and f(t) becomes zero at some point, you will always get M=1.0.
Mind, however, that this will not work if 2 * G = R = f(t) … that will produce M=2.0, then jump to 1 and back again – but that kind of case is impossible to get right unless G(t) and R(t) are known functions, and you’re able to do calculus to them. Even then it’s tricky, and I don’t know if you could solve it automatically.

I am not sure I follow. What is f(t) and this expression 2 * G = R 0 f(t) ….

whoops, typo! The expression was meant to be
2 * G = R = f(t)

What I meant is: If you have a gradient where G=R, and both go to zero, then my method will give you constant M=0'. But if you have another gradient where 2G=R(that is: R is twice as large as G, soG/R = 0.5, and they follow some gradient (symbolized by f(t), maybe f(x)` would look more familiar?), then G/R would also be constant and they would reach zero at the same time, but the result you’re getting for M would jump from log(2) to 0 at the location where they reach zero.

All that said, after reading through the other replies: If you’re dealing with RAW photos, then zero should not occur in the input data, and if it does, then adding an assumed (very small) black value to all inputs at the start should be the safest thing to do.

Why shouldn’t it? The example at Can you fix a heavy banding problem in my exposure? - #30 by snibgo has many pixel values that are zero.

Because the dark current is not supposed to be 0.

But in a low-light or high-contrast situation, the arithmetic of the light energy might mean that the recorded value should be 0.1 on a scale of 0 to 16383. Cameras don’t record floating point, so the camera rounds that 0.1 to zero. Sure, the camera could set a floor so zero is never recorded, but that would be less accurate.

And the fact is that cameras do record zeros. We might complain that they shouldn’t, but our software needs to deal with them.

I think you’re forgetting about the sensor’s noise floor. Prior to black point subtraction there will never be any pixels at 0/16383 (other than dead pixels). On my A7III the native black point is 512/16383 and the lowest any channel ever dips to is about 480/16383. Of course after black point subtraction any pixels below 512 will be zero.

1 Like

Thanks, @NateWeatherly.

When a value in your A7III dips below 512, I suppose that is because of noise generated by the camera. Is that correct?

True, some cameras don’t record values that are proportional to the light energy. Instead, they record values that are proportional to the light energy plus some constant. To get a proportional value (which we might need for arithmetic, so we can accurately add light values etc) we must first subtract the constant.

But some cameras, such as Nikons and whatever was used in the thread I referenced, use a constant of zero. Thus when hardly any light hits the sensor, they record zero.

Makes sense – but to come back to the original topic: In that case you should add some very small number to the whole data to remove any such occurrences, to make the data conform to the assumptions implemented in the equation, or at least deviate less from them.

Since adding some constant to each of the colour channels will affect the ratios of values between each channel, it should then also be clear that the choice of black level affects the hue in the darkest parts of the image. Thus, subtracting black level before raw processing has that kind of effect, too.

And that’s how I just worked out why subtracting raw black levels early in the process is not a smart thing to do, although I’d thought we were just talking about algebra here :slight_smile:

In general terms (i.e. for an arbitrary curve, not just spirals), assuming the curve is given by x(t) and y(t), where t is some non-dimensional parameter which starts at 0 and increases from there (in Thanatomic’s post, he used \theta for the spiral, which is also the angle in polar coordinates), the curve length s(t) can be computed like this:

{ds \over dt} = \sqrt{{ds \over dt}^2 + {ds \over dt}^2}
\Rightarrow s(t) = \int^t_0 \sqrt{{ds \over dt}^2 + {ds \over dt}^2} dt

In words: As t changes by some small amount, x changes by {dx \over dt} and y changes by {dy \over dt}, and the length of that very small piece of curve is \sqrt{{ds \over dt}^2 + {ds \over dt}^2}. Integrating (adding) that up from that start of the curve gives you length of the curve for any given t. Depending on what kind of equation you’re using, there might be an easy analytical solution, or you might have to do it numerically.

If you are doing it numerically, simply counting pixels will be inaccurate because two consecutive pixels on a curve are either directly next to each other (distance 1) or diagonally offset (distance \sqrt 2), independent on what slope your curve has. So if you want to know the curve length at some point t_p, it’s a lot more accurate to compute sample points along the curve for a large-enough(*) number n:
t_i = t_p\frac{i}{n} with i = 0, 1, 2 ... n
x_i = x(t_i) , y_i = y(t_i)
Then compute the distance between neighbouring sample points on the curve:
\Delta s_i = \sqrt{(x_i - x_{i-1})^2 + (y_i - y_{i-1})^2}

And from this, you can compute the curve length for your point by adding up all the \Delta s values:
s(t_p) = \sum_{i=0}^n {\Delta s_i}

The smart way to implement this is to choose t_p to be at the end of the curve (however far the curve matters to you), then choose n to be large enough(*), sample all the x and y coordinates, compute all the \Delta s_i, and then compute the cumulative sums of \Delta s_i, i.e. just the first value, the sum of first and second, the sum of the first three … and voilá, you have the curve length at every sample point.

To compute normals: Since you already have all the point coordinates, and assuming that curvature does not change much between the sampled intervals, you can compute the direction of the tangent vector using central differences:
x'_{i} = 1/2 (x_{i+1} - x_{i-1})
y'_{i} = 1/2 (y_{i+1} - y_{i-1})
This tells us how x and y coordinate are changing and so if we plotted a line starting from (x_i + 100 x'_i , y_i + 100 y'_i) and ending at (x_i - 100 x'_i , y_i - 100 y'_i), that would be a tangent to our curve. To get a normal line, we need to turn it by 90 degrees, which is easy:
x_{\perp i} = -y'_{i}
y_{\perp i} = x'_{i}
So a line from (x_i, y_i), to (x_i + x_{\perp i} , y_i + y_{\perp i}) would be perpendicular to the curve at point i.

To draw a normal with a particular length l, you compute the length of your normal vector:
\left| \left( x_{\perp i} \atop y_{\perp i} \right) \right|= \sqrt{x_{\perp i}^2 + y_{\perp i}^2 }
and scale the normal vector accordingly (divide by its own length, multiply with the length you want):
\left( x_{\perp i} \atop y_{\perp i} \right) \frac{l} {\sqrt{x_{\perp i}^2 + y_{\perp i}^2 } }

So, you can draw the curve at any point i with parameter t_i at coordinates (x_i, y_i), it has the curve length s_i, and you can draw tangent and normal lines at whatever length you like.

Done!

If you are using spirals or circles, then this can all be done on paper (as Thanatomanic has shown), although using cos() and sin() is not very fast for a computer. With some other curve types (splines, for example), all the derivatives above are really easy to do by hand, too, and work out to simple simple functions (they look something like ax^3 + bx^2 + cx + d). In the general case, it can get messy, though…

(*) large enough: depends on how “curved” your curve is. Generally, if the distance between two samples always is less than a pixel, you’re always safe, but if the curve is not bending a lot, you can get away with much fewer sample points. If your “curve” is actually a straight line, then n=1 is absolutely accurate

1 Like

Hi @Mister_Teatime, nice write-up, but I don’t think this solves @Reptorian’s original problem. The method you describe is very suitable for a single curve, but not to find the points ‘in between’ the arms of the Archimedean spiral, perpendicular to the arm itself.

Edit: I need to clarify. You do describe how to obtain tangential points to a curve. The tricky thing is: where does that line stop? The value of l can probably be found, but only numerically as well, as I have shown in my post.

In fact the arc-length of the Archimedean spiral is known analytically: see e.g. Archimedes' Spiral -- from Wolfram MathWorld
But again, I don’t see how that helps to solve the actual problem.

1 Like

Apologies for not having read through the whole thread, but what occurrences might those be @Mister_Teatime ?

Light transfer from scene to retina is expected to be linear with the origin at zero. Any non-linearities are introduced by non-idealities, mistakes or for convenience.

Ideally zero light intensity needs to be reflected in zero values as the very first step in the raw conversion process, otherwise key operations like white balance, normalized clipping not to mention some of the more advanced demosaicing algorithms and color transforms become messy and often introduce mistakes.

That’s also why astrophotographers do flat fields and some of the early Nikons used dedicated optical black pixels to determine a physical BlackLevel specific to each capture, subtracting it before writing data to the raw file, as I think @NateWeatherly mentioned. That took care of non-idealities like temperature dependence, inaccurate CDS, dark current and some DSNU. The downside to the Nikon approach was that it messed up histograms near the origin, which frustrated people needing uber-accurate mean signal readings there (a tiny percentage of users). More recently, possibly when 4-T pinned photodiode configurations became common, they apparently feel comfortable in the consistency of their BlackLevels (say +/- 0.5DN throughout the ISO range) and optical black pixels are nowhere to be seen.

Yeah, I might have gotten a little carried away :wink:

I had not read the original question to include having to find the intersections of the normals with the curve. That’s of course not easily solved when using some generic function to define the curve … if, however, you pick a curve which does not wrap around (within the bounds of the picture), you can just continue along the normals until you meet the edge, and get any type of colour gradient along any kind of curve.

I think the reply thread got messed up (or I did not hit reply at the post I was replying to … unthinkable!), so you can’t see what I was replying to.
@snibgo mentioned a raw file which did contain zero values, and my statement was in response to the occurence of such zero values in raw files – if and when they happen. I can’t of course comment on how they got there and if (or what) must have gone wrong to create them.

If I get it right, then the only way to get actually zero light on any pixel of your sensor is by preventing any light at all from getting to the sensor (keep shutter closed, or cover the lens and OVF), so any normal exposure should not have pixels which did not even get a single stray photon – this means the ideal raw file (of an actual scene) should not contain black pixels, even after black level subtraction.

Thanks for your explanations on dark currents and related issues! Following that, most cameras should never have black pixels (except if they’re dead, maybe?), but you might still get an image with applied black point subtraction, and depending on how that was done, some pixels could be “corrected” to zero – which would mean that the subtracted black levels would have been off.

I’m not doing astrophotography, but I did work with flatfields for image analysis in biology once, and what they do is:

  • correct for vignetting (i.e. if my scene consisted of perfectly even light, what would the camera produce?)
  • in the case I used it, also quantify the uneven background illumination of the light table we were using.

Following your explanations, I think you were referring to dark frames, not flatfields? I understand that dark frames are trying to capture the values introduced by the sensor and electronics of the camera itself in the absence of light. So I imagine (without practical experience) that if a dark frame was not entirely accurate (e.g. due to some minor shot-to-shot variation in the dark frame pattern, or some implicit simplified assumptions about the system), some particularly dark pixels from the scene frame could be over-corrected to zero (that is: to below zero, then clipped to zero).

Anyway, I started out with math, and ended up “getting” dark frames, so I’m feeling good now :slight_smile:

I agree until your final comment, which doesn’t follow from your previous:

– this means the ideal raw file (of an actual scene) should not contain black pixels, even after black level subtraction.

A camera can only record integers between zero and some maximum, eg 16383. Suppose the camera writes values that are proportional to the light energy, so zero light records a value of zero. The scene might have a contrast of 16 stops from darkest to lightest. So the darkest shadow should be recorded at 0.25, a value between zero and one, and closer to zero than one.

What value should the camera record here? Surely the correct, most accurate answer, is “zero”.

Similarly for lower-contrast scenes if the camera or photographer doesn’t ETTR.

“Zero” may cause problems in some algorithms, and they may choose to clamp that to a small positive value. That’s a different matter. “Zero” is a valid value from a camera, even when some light energy was received.

Just for giggles, I opened a few raws and looked at their minimum channel values, unadorned by processing. My Nikon D7000s all had 0 mins for all three channels, all the time. My Z 6 raws, however, have consistent channel mins of about 983, including a dark frame I shot for the purpose. Indeed, the Z 6 NEF metadata has a black field, value 1008, and I subtract that from the image or I get weirdness in the dark areas. Nikon D7000 has no such value in the metadata.

With apologies to the OP for the OT diversion, sensors are rough photoelectron (e-) counters. A 14-bit camera like the above mentioned Z6 can count a maximum of about 100k e- at base ISO, which means that it takes about 6.5e- for it to tick up one raw value at the ADC (100k/(16383-BlackLevel of 1008)), which means that ideally any perfectly valid signal of about 3e- or less should be recorded as 1008 (minus the BlackLevel = 0 DN). 4e- for instance will be clocked at 1009 (or 1 DN after BL subtraction).

However, the downstream electronics superimposes read noise on the output of the photodiode. While the signal is Poisson, so it can never be negative, read noise is hopefully Gaussian with a mean of zero, so it can push the signal to be ‘negative’, meaning less than the Black Level (1008 above). However, because of the symmetry in the normal distribution, if you take the average of, say, a 100x100 pixel area uniformly lit by a 3e- mean signal, the result will be dithered as a result and produce an accurate mean reading of 1008.46 +/- 0.01 or 0.46DN after BlackLevel subtraction.

Glenn’s D7000 on the other hand can’t pull this trick off because it did not carry the ‘negative’ values below the BlackLevel and therefore it crushed the symmetry of the bell curve truncating it to zero. Its raw values near zero illumination will therefore be biased to the right hence inaccurate. Which is why Nikon no longer subtracts the BlackLevel before writing data to the raw file.

HTH
Jack
PS some detail here Photographic Sensor Simulation | Strolls with my Dog

2 Likes

Thanks, @JackH. That helps my understanding.

@ggbutcher: Yes, some cameras don’t record zero at all, even in zero light.

If I had such a camera, I would investigate along the lines of Linear camera raw to understand what transformation is needed to get linear values, by which I mean values that are proportional to light energy, NOT plus a constant.

The required transformation may be a simple subtraction of a constant such as 1008, but I would want to test that for myself.