Human perception

We have a little egg vs chicken situation here.¹

Those exposure compensation numbers are based on the premise that the metering device is calibrated to those 18%. In reality all components play together and influence each other. For example if you would have used a developer that would overly develop the white areas, you might have to dial the compensation way down. The same applies when photographing with digital. Even more so, as digital will blow out and have no recoverable information beyond a certain point.

It is a nice practice to “calibrate” your camera for spot metering:

You meter off a large enough white area, photograph a series with changed exposure compensations and then look at the files in your raw editor with your preferred settings and note at which camera setting the white keeps too little information for your editing style.

Now remember that number and you can get spot on – pun intended – exposures.

For example: with my Nikon D500 and how I use darktable I have established a compensation of +2.7EV for the spotmeter onto the brightest area I want to have rendered with detail but very close to white.

Circling back to the main topic: this does not tell anything what that brightest part really is or how it should be perceived later by the viewer.


¹) in evolution the egg is always first.

1 Like

Might be very true - I have only calibrated closed loop systems and never the exposure meter itself.

If you think about it: a digital camera could be set to 99%, pick the brightest area from the sensor and expose to the right every single time, perfectly.

1 Like

I had to deal with log 49 years ago, but never again since then.

I’m concerned here with photography and the question of how to get the high contrasts of the subject and the camera sensor onto the screen and paper without everything becoming muddy and far too dull.

This question is important. So the question of which curve or other tricks are useful to achieve a good result.

Maybe others see it differently, but for me it’s not so important to understand the basics with the math.

I am now also prepared to not understand the log.

If you want, we can move on to the practice of curves. Please note that at least I don’t want to go deeper into computer science.

The exciting question is what I can do to treat the compressed contrast in such a way that at least the parts that are important to the image still retain sufficient contrast and brilliance.

A couple of points about contrast:

  1. When editing in the display-referred phase, we might ensure that at least one pixel is black and at least one is white. This is a simple contrast control.

  2. After that, we might want to increase contrast. This can be either be global or local (or both).

  3. Global contrast is changed with a tone curve. This will increase contrast for a certain range of tones, while decreasing contrast in other ranges. For example, a photo might have a bell-shape histogram, and we want to increase contrast in mid-tones, so we apply an S-shape curve which does that while decreasing contrast in shadows and highlights. The curve is steep in the middle and flatter at both ends.

Or a landscape might have a bimodal histogram, with mostly dark pixels (in the land) and light pixels (in the sky), with few mid-tone pixels. So our curve might be a double-S-curve, with steep portions in the dark land and light sky, and flatter portions elsewhere.

If we apply a curve at the scene-referred phase, we can apparently increase contrast in one tone range without reducing contrast elsewhere, because we don’t need to worry about clipping black or white. However, the final displayed image will need compression somehow. So when we want to increase contrast in some tones, we always need to consider also where we will decrease contrast.

  1. Local contrast might be done with different curves applied in different parts of the image. This might be controlled by hand-built masks, or automatic (segmentation) masks. Or it could be done with digital algorithms such as bilateral filter or guided filter, which each can blur (or sharpen) areas but not edges.
2 Likes

You specifically asked that a new thread be created to talk about background, and not about curves. I think the questions about curves should be discussed in the original topic.
It is also possible to talk about curves in general, but then it’s going to be about maths, after all.

1 Like

I tried to take these concepts into account by using Ciecam (Color Appearance Model), since 2012…and now with Cam16.
With my interpretation :“Trying to take into account by software, the physiological aspects due to the perception of the eye and the brain”.

Some of the effects taken into account:
https://rawpedia.rawtherapee.com/CIECAM02#About_CIECAM02
Simultaneous contrast, Hunt’s effect, Steven’s effect, Helmholtz-Kohlrausch’s effect, Chromatic adaptation, etc.

An example taken from the list:

You can get an overview of what Ciecam is with the tutorial (which should probably be updated a little)
https://rawpedia.rawtherapee.com/CIECAM02

I recently made several tutorials on the subject (directly or indirectly)
https://discuss.pixls.us/t/paris-olympic-games-pont-alexandre-iii-tutorial/41895
https://discuss.pixls.us/t/another-tutorial-color-appearance-truck-under-a-tunnel/41947
https://discuss.pixls.us/t/scene-reffered-and-display-reffered-part-1/42076
https://discuss.pixls.us/t/scene-reffered-and-display-reffered-part-2/42087
The 3rd part, the summary will soon be available.

Concretely there are 2 modules present in Rawtherapee:

  • Color Appearance & Lighting (Advanced tab) - complete module which is complex for a beginner
  • Color Appearance (Cam16 & JzCzHz) (Local Tab) - in Basic mode, this module is quite simple and allows you to address (and solve) general colorimetry problems, including for high dynamic images (25 Ev)

Jacques

3 Likes

I am not sure that thinking about it this way is helpful. At the end of the day it is all about light hitting the sensor or the retina, and whether it comes from a light source or a reflecting surface is of secondary importance, once we know the intensity.

Similarly, when it comes to specific curves, such as filmic or sigmoid, I think about practical considerations instead of trying to derive them from first principles about human perception. The photo industry ended up with a lot of very useful ways to map light into paper by chemical film, but let’s not forget that this took more than a century of experimentation.

For digital signals, we can be more flexible with post-processing, but practically most people want something that minimizes the work so they can focus on other aspects of the image and/or postprocess their photos quickly. Sigmoid may just be a sweet spot.

Finally, I think that we the retina and the brain is doing something that is not unlike local contrast enhancement (I am not an expert and maybe someone can link references). For digital photography it is fine to decouple the global mapping from this, but let’s not forget that film did this too and it can be emulated digitally.

3 Likes

When our theme now is “perception” I believe its pertinent to point to that our understanding of the role of perception lately has changed somewhat within our current understanding of how the brain functions at large.

For more than hundred years our main concept of the brain has been like a reactive information processor: We receive some sensory stimulus, which our brain then subsequently processes, and from this the brain command new acts.

For about the last decade, however, a new picture is emerging and beginning to take foothold in psychological research: That of “the predictive brain”. Related to this there is now also much more focus on the fact that for all the nerve connections that goes from our sensory neurons to the brain, there is about the same number of connections going back from the brain to our sensors. This latter fact has largely been ignored for most of the era of psychological research.

What does this mean?

Proponents of “the predictive brain” argue that the brain’s overall role is to aid our body in performing a next act in a way that likely should increase our survival. To achieve this, it’s too slow just to react on incoming sensory signals. Rather the brain constantly constructs images / spatial models of our whereabouts (also in abstract ways like our social relations), and from that it predicts what is likely the best next step. Our perception then serves to validate our predictions, and if it doesn’t, a revision of the brain model is induced. (It’s also argued that this mode of brain function is a more cost effective way for the brain to work, which of course is an evolutionary important aspect. Less eating necessary - our brain currently consumes about 20 % of our food intake.)

It also probably mean that our perception of our sensory stimuli are likely to be modified by our understanding of our surroundings and by our mental states more than we previously believed. It has e.g. been demonstrated that even such a fundamental “mechanical” function as regulating the aperture of the eye’s iris, is not determined alone by the intensity of the light reaching the eye, but that the size of the opening is influenced by what we believe is the intensity of the light.

Much of our visual response testing – and much of psychological laboratory research in general – is made within constructed research test environments where we expose participants to one or a few stimulus and focus on this in isolation. But this raises serious questions of the ecological validity of any knowledge we think gained from such tests. Because our brains and our senses normally work within a much richer environment where input from one sense is combined with others, with our previous experiences, our understanding of current context, our moods/emotions etc, all mixed together. In such a setting e.g. our perception of light strength is influenced e.g. by our perception/believes about distance to the light source – and our perception of distance is dependent on several physical aspects, not the least by moving head/body within those surroundings, and on and on an on … it’s all connected. Furthermore, attention is in general an important factor that influence our judgement of the magnitude of sensory input.

Take a look at CIE’s definition of “color, perceptual”|“perceived color”: “characteristic of visual perception that can be described by attributes of hue, brightness (or lightness) and colourfulness (or saturation or chroma)”,
and then at Note 1: “Perceived colour depends on the spectral distribution of the colour stimulus, on the size, shape, structure and surround of the stimulus area, on the state of adaptation of the observer’s visual system, and on the observer’s experience of the prevailing and similar situations of observation.”

We now know that all the three color dimensions, hue, saturation, and brightness, of color stimuli, as well as their interactions, have various effects on the emotional state of the observer. Since we know emotional state have effects on many perceptual and other cognitive aspects of our nervous system, we should likely include “mental state” into that Note 1. But as Freud once noted, (at the time he still was trying to establish a scientific psychology before he gave up due to the then lack of instruments): “Es ist nicht bequem, Gefühle wissenschaftlich zu bearbeiten.” - and for many reasons it isn’t much easier today to make research were the effect of an extensive set of emotions should be taken into account, (including because after more than 150 years of scientific psychological research there is still no agreement what an emotion is or what types of emotions there are … ).

We never perceive a pixel alone, but as part of the total image with all its variation in lightness and color, surrounded by its environment, including ambient light, and as part of our total mental state.

I see the need for getting as good an understanding of our perceptual responses as possible, and the need to create image processors with e.g. orthogonal controls with well-behaved response curves that can lead to us having tools being as accurate as possible for e.g. reproducing other images, (but we need to take into account that the perceived colors of a painting is also the result of the physical aspects of paint and light from various angles falling on it …), or doing product or portrait photography and the like.

For the rest of us who try to “create nice pictures”, we might find some relief in the fact that in seeing an image our eyes just make a few point readings and fill inn the rest as fits, something painters (and magicians) have known to “trick” us - we are deceived in the feeling of really seeing a wide picture – and from the fact that I’ve never heard any member of a photo jury declare that if it wasn’t for some deviating nuance in hue or chroma, a picture would have won an award.

TIP: In this latter respect I will make a proposition to spend some more time on the orientation module, rather than on the intricacies of the color balance module.
Why? Because most images will likely be perceived to be “better” if it complies with basic composition aspects relating to forms and lines and their contrasts and balances, (see any photo book on composition for details). However, part of our perception process is to find meaning in what we see, and in particular if there are faces or other elements of humans and their activities, this is likely catching our attention. By turning images upside down, there is in most images little immediate meaning left that can catch the attention, and hence we are better positioned to see and judge the image at its basic level of forms lines, etc.

4 Likes

Hello @kofa,
It is an exciting question how to achieve a higher contrast at a desired point in the image.

How do you determine in the raw development where the contrast should be higher, but at the expense of an even lower contrast in the other areas?

Of course, I am primarily interested in ART because I want to work with it. I see that you can do this with Sigmoid with Skew and a little with White point.

What other options are there for increasing the contrast at a certain point? I think you can do anything with the curves, but it can quickly look artificial if you don’t do it very sensitively.

I’m curious.

There is a critical misconception in the original post: that we need to encode a log image in order to look natural. This is false.

The most natural encoding is linear. If we record linear light, and display linear light, it would look perfectly natural. Our eyes are doing the log encoding. It is our perception that is linearly higher for a multiplicative stimulus, but the stimulus is still linear. It does not need to be added digitally.

The problem in photography is, we do not have displays or printers capable of displaying a linear scene. A sunset may contain as much as 20 EV of dynamic range. That is a 2^20 range of linear light energy. A staggering contrast. Of which our cameras will record as much as 12 or 14 EV. But our displays will be able to show 8 or 10 EV, and a print 4 to 6 EV.

It’s this reduction in contrast that requires the sigmoid curve. We need to map the tones of the recorded light onto the limited dynamic range of the display or print. A naturally lossy process. But there’s nothing “natural” or “perceptual” about it.

We have learned to interpret a small two-dimensional projection of a scene (what we call an image), and can derive emotional meaning from that. Non-technological societies can not do this. They do not see meaning in a photo beyond a colorful piece of paper. This is learned behavior. And as such, we have learned that white parts in an image represent bright things such as sunlight while black parts represent darkness. And that the tones in between are an artist’s rendition of critical detail. This is what we’re trying to achieve with our tone-mapper. Map the critical parts of the scene to middle tones, while maintaining a semblance of brightness and darkness otherwise.

8 Likes

Sorry, I did not mean to imply that. I agree with what you wrote above.

1 Like

All pixel-wise operations can be represented as curves: on the x-axis, the input, on the y-axis, the output.

Then there are operations which act on areas. I believe ART and RawTherapee call this ‘contrast by detail levels’, but I don’t use them actively, so I’m not sure.

However, did you not ask for this topic to be opened because you wanted background info? The original one was about ART; maybe that would be a better place to ask ART-related questions.

1 Like

Low contrast

High contrast

If tone curve of the image has steep slope (= allocation input range narrower), the image has high contrast.

So…

High contrast: Much steep = narrow input range allocated wide output range

Low contrast: Less steep = wide input range allocated narrow output range

3 Likes

Thanks, @yasuo, for putting some curve tutorial into the discourse.

Understanding tone manipulation is about understanding transfer functions, and understanding transfer functions is about understanding curves. A curve is really just a graphic representation of a transfer function; if you take any tone operator, filmic, sigmoid, log, etc, and plotted its output from an input of, say, 0.0 to 1.0, you’d get a curve. Same sort of curve you mess with in the control-point curve tool. Difference between the two is that the named operators have some equation at their root, control point curves have a point list and some spline equation to smooth the plot between the points.

And, as @yasuo points out, the slope of the curve defines the contrast applied (or not) to that segment of the tone distribution. The only thing I’d add to the graph would be the straight line between the bottom-left and top-right, which represents the “identity” of a curve, where each input value = the output, or ‘no-change’. Then, you get a sense of which parts of the curve are increasing the tone value, and which parts are decreasing it, based on whether they are above or below the identity line…

2 Likes

@bastibe, thanks for articulating what I couldn’t find words for…

What we need to consider in our photographic endeavors is end-to-end how the tone distribution is manipulated to make the final rendition. it’s our rendition media that we’re accommodating, not our vision. The whole gamma thing is really a legacy of the cathode ray tube, whose tone response was decidedly non-linear, and our workflow ever since has been whipsawed to accommodate. LCD screens could provide linear response, but they are usually skewed to CRT gammas to accommodate legacy sRGB encoding.

When I process a raw in rawproc, the first operation is to just ingest the raw data. I can select that first operation for display, and the image looks quite dark. But, not as dark as it really is because the display pipe is pushing that raw data through the display transform, where the color side is meaningless as each pixel only has one value, but the tone side is being applied. And, its effect is visible, and indicative of what the display requires.

I then add operations on-by-one, until the display rendition of the last tool in the chain looks nice. The tone curve, at the end of the chain, presents whatever additional tone manipulation is needed, usually a lifting of the shadows to accommodate my highlight-weighted exposure; the major lifting already done by the display transform. Again, all to accommodate the rendition medium. Stopped using middle-gray exposure some time ago…

So, when I export a JPEG, I transform the export data and embed to a sRGB colorspace and tone. This makes the image presentable on a non-colormanaged setup, and also provides the information for a color-managed setup to do an appropriate transform. A bit of a crapshoot, but provides the best chance for my rendition to look like I think it should to others.

Okay, a bit of a story, but what I want to get across here is that we are working to the vagaries of our rendition media, not necessarily our perception…

4 Likes

Yes, the curves determine where the contrasts are high and where they are low.
Do you know of any other ways in ART to increase the contrast at certain tonal values - always at the cost of other areas, of course.

That is correct – but isn’t it the log-like behaviour of our perception that lets us ‘get away with’ such extreme compressions?

That last bit is interesting. I have never read about such experiments. Do you have a source?
I have read about for example perspective: people spending their whole lives in dense forests, like jungles, supposedly don’t experience perspective as we do; when taken to the plains, they are amazed that distant animals ‘grow’ when approached. However, I find that suspicious: even in a forest, you have tall trees, and birds that look small when perched up high in the tree and larger when you cook them…

3 Likes

You’ll need rawproc on a mobile computer to do this… :laughing:

Say, you’re looking at a sunset. Take a picture of it. Import it to rawproc, assign the camera color space, black-subtract, demosaic, and scale the data to the black-white boundaries. Depending on how you exposed the shot, the sun may be blown, or the shadows may be night-black. So, why doesn’t a linear rendition on a monitor look the same as the actual scene? Mainly, because you’re looking at a device with its own limited tone response and gamut, with a linear image shoehorned onto it. The behavior of your vision is working with the display, not the actual scene. So, I don’t think it’s that our wildly complex vision system lets us “get away” with tone compressions, it’s that we need to do that to accommodate the limited range of the medium.

When you look at a picture, you’re looking at an oh-so-coarse-and-compressed rendition, not the actual scene…

1 Like

Wehre can I read about this?

1 Like

When we fit the brightness range of the image onto the display by linear scaling, we (generally) darken everything. I think it’s important to keep midtones intact, so they look like the average surroundings of the display, as a reference. But that means we need non-linear compression, usually at both ends of the range.
If the resolution of the display were infinite, despite the limited max. brightness, looking at the display in complete darkness would resemble the original scene, I think (although at extremely low light levels, our vision also changes).

2 Likes