I enjoyed the read. Thanks for making it through on just one coffee! 
1 Like
Sometimes you need to clip. For example when using a Hald CLUT.
@Elle, well, I was just thinking about the raw data pipeline: applying black and white point, CA, impulse noise and other corrections, white balancing, demosaicing, noise reduction, luminance recovery, etcetera. After converting the raw data to a color space, I’m not sure if it’s feasible at all.
We should totally use that from now on.
2 Likes
I would like to add the word “scooch” to the technical lexicon. With respect to curve control points, it refers to their slight movement to achieve a particular image transform. For each scooch, I’m sure there’s a polynomial somewhere… 
1 Like
Negative (more generally, out-of-gamut) values confuse the heck out of me, but they are a sad fact of life.
They can arise even when in monochrome, even when not messing with profiles. For example: suppose an image has only three pixels. They are black, white and 25% gray. We resize the image to be twice as large, six pixels. What gray levels are they?
It depends on the resizing method we use. The default ImageMagick HDRI (floating-point) resizing gives:
convert xc:black xc:white xc:gray(25%) +append -resize “6x1^!” txt:
0,0: srgb(-3.08924%,-3.08924%,-3.08924%)
1,0: srgb(24.6607%,24.6607%,24.6607%)
2,0: srgb(76.497%,76.497%,76.497%)
3,0: srgb(83.3832%,83.3832%,83.3832%)
4,0: srgb(44.1264%,44.1264%,44.1264%)
5,0: srgb(22.6831%,22.6831%,22.6831%)
That’s squirrelly.
What has happened? Think of a graph that passes through 3 points: 0, 1 and 0.25. Resizing will re-sample the graph. The algorithm used by IM assumes the graph is a curve, rather than a pair of straight lines. So the resampled points can be outside the values bounded by the inputs.
Another example: unsharp masking can push values out of gamut. This is obvious if you think about what USM does, but here’s an example. Make an image that is black and white only, 3x3 white pixels surrounded by a black border 3 pixels thick:
convert -size 3x3 xc:white -bordercolor Black -border 3 -format “MIN=[fx:minima]\nMAX=[fx:maxima]” info:
MIN=0
MAX=1
Unsharp-mask it:
convert -size 3x3 xc:white -bordercolor Black -border 3 -unsharp 0x2 -format “MIN=[fx:minima]\nMAX=[fx:maxima]” info:
MIN=-0.197727
MAX=1.75652
Eek! Values now range from -19.7% to +176%.
Yes, OOG values (especially the negative ones) will cause some algorithms to fail, eg:
f:\web\im>%IMDEV%convert xc:gray(-10%) -evaluate Pow 2.2 txt:
0,0: gray(-nan%)
Double-eek! The pixel is not-a-number!
Software routinely avoids the problem by clipping, but this is a sledgehammer technique that destroys data. We can do better, I hope.
@snibgo This becomes increasingly apparent as I use G’MIC and IM more regularly. It confuses the heck out of me as well. I suppose it is a case by case but how have you tackled the problem besides clipping? No one seems to be ready to discuss this in practical terms whenever I bring this up but we have to start somewhere.
I think it’s just the inevitable consequence of manipulation. Almost every operation inflicted upon digital data requires an implicit decision to ‘bucket’ the result into an adjacent discrete value, losing accuracy. This starts with the sensor ADC.
The effects described in earlier posts are new and interesting to me, compelling me to reconsider some indiscriminate image manipulation. Well, maybe not, the resulting images appealed to me, and the loss of data didn’t take anything from that. Might be a different story if a museum came to me with a proposal for a showing, all images in 6’ x 8’ (that’s feet) size. Fat chance of that…
What I decided when I took on a floating point internal format for rawproc was to let the over/underruns accumulate, and clip for output. This intuitively seems prudent, as it’s the data I’ll see that matters to me. Indeed, I’ve had some images where I’ve had to lose some clarity in light sources in order to properly develop the rest of the scene, pushing the light off the visible range. Can’t afford a D850, for the dynamic range.
Passing unbounded image data to, say, GIMP opens the opportunity to recover some of the spill with tools I’ll never be able to implement. That’s what I propose…
My current strategy, if you can call it that, is to favor manipulations that don’t deviate from the bounds as much. For me, +ve values are scoochable but the -ve OOG ones are squirrel-aggressive.
1 Like
We need to write a paper… 
maybe (in fact, likely…) I’m being naive, but if you don’t know how to handle out of range values, why don’t you just leave them untouched? i.e.:
-
apply transformations that are identity for values out of the [0,1] interval, and
-
clip upon reading the pixels instead of when writing (i.e. if you are combining the pixel with its neighbour, just pretend that it is in range)
why wouldn’t this work (besides introducing overhead if you read more than you write – which is likely the case)? experts please enlighten me 
Thanks for you input. Compared to my level, you are all experts 
.
Software could give users a choice about out of gamut (OOG) values, such as:
-
Never clip or tame OOG values.
-
Always clip after every operation.
-
Always clip before every operation that OOG would cause to fail.
-
As (1) but apply a non-destructive process to put OOG inside the box.
-
As (2) but apply a non-destructive process to put OOG inside the box.
Non-destructive processes (“scooching”?) include applying a gain and bias (eg ImageMagick’s “-autolevel”), and methods I show in Putting OOG back in the box. Other methods are possible, eg the four ICC profile intents (only two of which have a precise definition).
This choice could be a generic user-preference, and/or selected for individual operations.
I have limited time to give to this at the moment, but some thoughts…
-
@snibgo, in your 3 pixel example I can see I think why applying some maths in the upscaling would lead to negatives, but it makes no physical sense if we’re talking black and white and colour spaces are not involved. Of course with lots of these operations, the boundary conditions have to be dealt with, and with only 3 pixels in a line (if I understand correctly) that’s much more boundary than interior!
-
I’ve not seen anyone address the point I made recently in the other thread where values are multiplied by 2 and then clipped; as opposed to just multiplying by the max suitable value and not needing to clip.
-
Elle’s site has warnings about unbounded operations ( Limitations of unbounded sRGB as a universal editing space ) but on a quick read these problems perhaps go away as long as the working space is big like prophoto (and better still if linear gamma also used). Though I have yet to try linear g. properly.
-
Re. further discussion of this and “white paper”, maybe an approach is to limit it initially to photos intended to be realistic / natural-looking, thus perhaps simplifying the issue by not considering some of the more complicated operations, e.g. ? blend modes that are rarely needed? (though I don’t know what most of them do in any case!). Consider the complication raised where two negative values are multiplied hence a plus. I can understand multiplying an image by say 1.6 to increase contrast, sort of, but 2 negatives, i.e. multiplying one OOG colour by another OOG, what does that mean in reality?!
sounds good, does this mean it scales all the time (in fl.point) to keep everything in gamut? (haven’t tried IM)
- Say you’re editing / RT-ing in prophoto and want to see how it’s going re. gamut. How about a nice new OOG tool where you say what space you’re thinking of converting to, and it highlights the OOG parts, but in addition you have a slider, which at one end says “show everything that’s OOG” and at the other “Just show the most OOG part”. Like soft-proofing but more generalised. It shows where the problems are then it’s up to you what you want to do about it.
My 3-pixel example was chosen for simplicity. The same problem can occur with mega-pixel images: if it has an area of high contrast, a USM can increase contrast to push values OOG.
Yes, operations can be designed to never generate OOG values. This would mean an unsharp mask would be adaptive, with varying effect depending on how much it would push values OOG. That’s not a trivial problem.
Sadly, no. The issues I show apply to any bounded working space. No matter how large it is, there will always be operations that can push pixels OOG.
Of course, for many purposes we just don’t care. If we just want a pretty picture and a handful of pixels out of a few million get clipped, so what? But how about 1%? Or 10%? Or what if we are preparing images for use in a larger project, where damage by clipping might be magnified?
The ImageMagick “-auto-level” operation is called only when the user asks for it. So I can call it after the USM, eg:
convert -size 3x3 xc:white -bordercolor Black -border 3 -unsharp 0x2 -auto-level -format “MIN=[fx:minima]\nMAX=[fx:maxima]” info:
MIN=0
MAX=1
In this case, “-auto-level” pushes all values inwards, so they span just 0.0 to 1.0. Aesthetically, this is often a bad idea. Imagine a photo with all pixels in gamut except for one pixel that is at 200%. “-auto-level” will halve the brightness of all pixels.
Yes, I think we need better OOG tools. And my point in these posts is to highlight that OOG occurs frequently, but we just don’t notice because software silently clips the issue away.
2 Likes
rawproc might be the basis for a case study in what tools do. Without considering the things we’ve discussed here, I simply let all my tools work the data, and what will be, will be. Clipping doesn’t occur except for 1) display, and 2) output to one of the image formats, currently JPEG and integer TIFF, soon to add PNG and unbounded floating point TIFF.
I have a reference image I use for testing, slightly underexposed (in both ETTR and JPEG terms) except for a glaring locomotive headlight, which clips even in the original linear capture. I have a image stats dialog box which I can call for any tool in the chain, reports among other things min/max RGB values. i was somewhat surprised to see maxes around 2.0 in some channels on this image after applying a series of tools, but it then occurred to me I was just pushing more of the headlight into oblivion.
Currently, all looks well in display because the image pulled for display is clipped to the display range, but I’m considering a selectable “blinkies” mode (don’t worry, I in no way am going to make pixels blink, probably some cyan/magenta coloration) to show out-of-bounds regions of pixels.
Even then, I’ll not consider automatically clipping or recovering OOB values in the tools, save maybe to implement a separate ‘recover’ tool. I’m probably not smart enough to do that, however, so I look forward to the opportunity to save to unbounded floating point TIFF, then open that image intact in GIMP to do recovery magic…
The white paper this thread reminds me of the most is on audio technology, the dbx type IV a/d conversion system where input is limited logarithmically after a linear segment in order to extend headroom before reaching logical clipping. It is a form of dynamic range compression, and introduces aurally pleasing artifacts (third harmonic distortion) instead of a hard clip.
I like this because you can see very clearly something is happening and deal with it. Suppose the tools in RT or Gimp had an option to do the operation either with clipping or auto-level, and you could flick between them and see the difference. Or at least have the ability to do the operation with auto-level for some tools. If you were increasing contrast, I suppose it would work as expected until pixels hit the limit, then the image would mostly go darker and less contrasty the more you increased the slider. This would indicate an OOG or highlights issue and then depending on its importance in the photo, you could decide to do the contrast adjustment with clipping, or e.g. do a local edit and reduce the saturation of the offending parts (or the whole image), etc.
@RawConvert Unsure whether it is wise to expose an auto-level v clip option, etc., to user-facing tools. That would over-complicate and clutter the GUI. I would rather have the devs make most of those decisions for us. Also, certain things only work as intended with clipping, as least visually. If you are interested in that stuff, I suggest you try using G’MIC and / or IM, if you aren’t already doing so.