How is an image’s tone mapped to display brightness (cd/m2)?

What is the relationship between an image’s tone and display brightness cd/m2, as opposed to white?

Does ~100%/white get mapped to the ~max brightness regardless of the device capabilities (80/sRGB, 500, 1,600 cd/m2) based on tone curve(EOTF)?

I think have a reasonable grasp of the basics of color spaces (primaries, WP, and tone response curves) but don’t know enough about the practical/implementation aspects.

Motivation. It seems the capabilities of phones, tablets, computer displays and TVs are evolving to both bigger color spaces (*P3) and HDR/brighter output (500, 1000, 1,600 cd/m2). Also there seems to be a more capable output format with JXL. I know this is in early stages but would like to understand.
Thx

2 Likes

Well , actually one of the true changes about HDR formats, is that they are supposed to be ‘in absolute brightness’, not relative brightness.

Old formats basically say 0% is ‘as black as you can’, and 100% means 'as bright as you can '.

But that means it is a different brightness on every device.

HDR formats describe which values map to which cd/m2 brightness.

So, eventually (i think we are not there yet ) is that those controls are meant to control where to put certain brightness values in certain value ranges.

Or, basically, it controls how much tone compression there needs to be done, because this can (and will) differ on output format in the HDR era, without messing with the rest.

What the controls do now, i dont actually know :see_no_evil:. I think they control where the max white point is placed. And 100 cd/m2 means 100%, and 150 cd/m2 means 150% , etc…

But this is probably all going to change when (or if) a proper HDR output pixelpipe is made.

2 Likes

That’s only correct for one of the HDR formats in use - Perceptual Quantiser. This describes what brightness each pixel was on whichever mastering display was used. There are then a number of metadata formats to tell the consumer display how to change the mastering display to your monitor - HDR10, HDR10+, Dolby Vision, HDR-SL1… (Not all displays do all metadata formats, so each manufacturer also has their own fallback version)

Hybrid Log-Gamma and other proprietary formats are scene referred - 0% is display black, 100% is display peak and there are published transfer curve adjustments for maximum screen brightness and ambient viewing conditions in the spec. No metadata is needed and you don’t need a monitor matching the capabilities of the mastering display to see it as intended.

Here’s Chris Lilley’s (of W3C and PNG fame) overview:
https://svgees.us/TPAC2020/intro/#PQ-HLG

Part of the problem is also that darktable (and many other SW) is based around ICC, which does not have this absolute brightness concept and only works on a 0-1.0 range assumption (1.0 being the media, i.e. display, printer, whatever white/max). It’s only recently that the ICC started looking into HDR concepts.

HLG is a weird one because it wants to be a hybrid. Don’t actually know exactly how it works. I think it’s basically SDR but just a different gamme curve to handle highlights or something?

HDR10 and the like do basically contain a curve that maps to cd/m2 or other ‘real world brightness information’. Also, the reason why viewing HDR content will see your brightness control being ‘taken over’, you are seeing something that your display is managing to do justice with all it capabilities, so it will crank the brightness.

Still, every screen is different, so there is always some sort of tone-mapping happening, but it’s now happening on your screen, instead of earlier in the pipe (the realistic thing is that it’s now happening everywhere of course, it’s not so black-n-white). Your screen knows it can only do 500 nits, for instance, but it will decode the signal (picture / movie) and know ‘this pixel is actually 1000 nits’. Now, how that screen is going to handle a 1000 nit area, knowing it can only do 500 nits, that’s up to your screen. And also why you have good HDR experiences and bad HDR experiences, depending on the screen capabilities but also the screen software :).

The ‘absolute brightness’ I am talking about is not always in the metadata, it’s also in the tone-curves used in each HDR standard. It comes out to almost always the same: The movie player now knows something about the intended brightness of that pixel. While in the SDR area, we only know ‘100%’, which tells us actually very little.

The metadata of some formats help the screens in doing better tone-mapping, by also telling ‘the movie player’ what kind of brightness it can expect. Sometimes static for a whole movie, sometimes dynamic in embedded metadata. But it still stands that the player now knows something about absolute intended brightness, and needs to do some tone-mapping itself (taking into account the hardware capabilities of the screen in question).

This is why AP started thinking about filmic and scene-referred.
If you tone-map early in the pipeline (like all other tools are doing), you are basically making a choice to limit it to a certain absolute brightness, and starting an SDR output, by doing it early in the pipeline. You then add modules / algorithms to tune up that image after it.

If you then want to make an HDR version of it, you basically have to redo the entire edit.

By doing the tone-mapping as last step, and do all other edits on ‘real world brightness information’ (simply explained), that means you can simply tweak the tone-mapping to another target device without having to do the edit again and while preserving the same look.

Since there is no HDR output yet, this is all just theoretical. And also why - what kmilos is saying - Darktable in the end still thinks about 0% and 100%, and the target display cd/m2 value is now a bit of a weird one. It controls the tone-mapping target, but the real-world usage will change I guess once there is an HDR output pipe.

ICC also have a much wider user base than ‘creating nice looking pictures’ that you get in Motion Pictures/Broadcasting/Artistic photography.

So whatever solution is eventually come up with must work for medical imaging where colour fidelity is very important, print media where you can’t have specular highlights etc. etc.

PQ and HLG came out of two different industries.

Motion Pictures where you shoot raw, have lots of control of lighting, lots of post-processing time and quite a lot of control over the viewing environment.

Live broadcasting where you have no lighting control, no post-processing time and the output is viewed on a huge range of screens under lighting ranging from sunshine to pitch black.

If you want to read a bit more in to it, then EBU Tech Report 70 is a good start. https://tech.ebu.ch/docs/techreports/tr070.pdf

Again from the W3C slides, you may find the web going towards HLG as it does not need the re-rendering step to view in different environments (the curve adjustment is done according to a formula in the display which knows its own peak and ambient brightnesses): WCG and HDR at W3C

PNG and similar formats are actively adding HDR. I think if you’re shooting RAW, then the choice will come down to what is easiest.

No metadata is needed only because the max brightness supported (nominal peak luminance) is 1000 nits (REC 2100).

If you have a 2000 nits hlg footage then it wiil displayed incorrectly in every TV/monitor.
(No metadata sucks!)

1000 cd/m2 is the reference viewing environment but, in the same way that 100 is the reference for bt709 and 80 for sRGB, it does not have to be shown at that level. They all adapt to the monitor or TV.

Note 5f in 2100 gives the maths for a different brightness screens (“For displays with nominal peak luminance (LW) greater than 1000 cd/m2, or where the effective nominal peak luminance is reduced through the use of a contrast control, the system gamma value should be adjusted according to the formula below”). Bt.2408 has section 3.2 describing adjustment for different ambient lighting conditions.

I’ve seen HLG displayed on TVs with peak brightness between 500 and 4000 and in a cinema - if the correction is made it looks similar in the same way HD TV looks similar at different brightness. Obviously the brightness doesn’t match - but being relative, it’s not designed to.

It’s also available in Dolby Vision and is used in quite a lot of portable devices.

Not proprietary, a published standard by ARIB.

https://en.wikipedia.org/wiki/Hybrid_log%E2%80%93gamma

Also, I don’t understand how any curve that lifts the image data from linear is “scene referred”… ???

1 Like

The other proprietary formats I meant were those from camera manufacturers like S-Log 3.

The image is scene-referred because it conveys the light incident on the camera sensor rather than the output of a specific display. For more information: https://www.color.org/hdr/07-Nicolas_Bonnier.pdf

The raw data from the camera indeed, but once any of these tone curves are applied to it, its values are no longer referred to the scene.

It depends on if the function is mathematically reversible or not.

Indeed, but in a real workflow, why would you want to do that?

I’m just not a fan of slinging the data around any more than is necessary to get the final rendition. Yeah, I know of things like RT’s pre-demosaic white balance that is backed out so the UI tool later in the pipe has its say, but such is a compromise to keep the user interface responsive.

In rawproc, I put white balance before demosaic, and I live with the delay in making changes to it.

In vkdt, it’s so speedy-quick white balance could be put anywhere and the delay from making a white balance change to display update would not be noticeable.

Because you don’t have the luxury of storing 32-bit floats for example.

and to convert between one type of camera signal to another.

16-bit integer is sufficient to retain most general purpose cameras’ measurement fidelity.

I’m not seeing a practical use case. Not dismissing the need, just don’t see it from my knothole…

Bear-of-little-brain here thinks, if linear data is important, I’ll maintain that linearity through the operations that need it, rather than caving to a workflow that inserts a non-linear transform before those operations. That implies shooting raw and retaining that original data, then working out a workflow that pays homage to that need.

And yet, not sufficient for HDR (e.g through in-camera combining of multiple exposures, conversion gains, etc.). Even so, 16-bit takes 60% more space than 10-bit. For video work this matters, believe it or not…

Sure. But storage and transmission do not need it. In fact, linear signal is extremely wasteful there.

1 Like

One of things I enjoy about this forum is a lot of smart/knowledgeable people interested in the fundamentals. Thanks for you responses.

Although I did read a number of things on HDR, I frankly did not give this much thought/reflection before asking. Clearly one does not want every white to be blasting them in the face with 1,600 cd/m2. I expect we will need a more sophisticated Filmic that can somehow allow us to differentiate between diffuse white and high/specular brightness areas.

This slide deck (https://www.color.org/hdr/10-Hamzeh_Issa.pdf) compares user experiences of SDR with HDR. It highlights that the benefits of HDR are reserved for certain scenes with higher levels of brightness.

This slide deck (https://www.color.org/hdr/11-Max_Derhak.pdf) proposes changes to ICC 5 which to my knowledge is not in mainstream use yet.

Due to an incident with a big cup of tea and my old MacBook, I now own a MacBook with an XDR display, so thought XDR/HDR were closer than they now seem. Seems a bit of a first mover problem, building capable devices vs. standard formats.

Based on my latest reading it now seems that AVIF format is preferred (implemented) by several if the biggest companies (Apple, Google, MS…) six months ago I would have thought jpg XL was the inevitable. I expect a battle between those making devices that want the best rendering to show off their HW and those storing/delivering the images that want small files will ensue.

Apple has employed Presets (image) that define the display mode/capability which seem reasonable but add another twist on how it would respond to an image with profile/metadata. So preset Photography (P3-D65) disables brightness adjustment, nighttime, True Tone.

It will be interesting to see how all this unfolds.

I expect we won’t use anything like filmic in the case of HDR: filmic is designed to tonemap scene-referred data with a potientially very large dynamic range to a limited range for e.g. paper prints.
If you prepare images (or video) for an HDR display, you don’t need that tone-mapping, at least in theory. That is not to say you may not need a log transform (to limit the amount of data to be transferred/stored) or a dynamic range control, but it will not use the same design principles.