HDR, ACES and the Digital Photographer 2.0

Software
1

After reading the ACES Primer (thanks @Elle for pointing us to this interesting reading!), I have started to consider how all that could be applied to digital photography.

If I understand correctly, one of the problems that the ACES workflow tries to address is how to properly render a scene-referred content (which is by definition unbounded and therefore “HDR”) on different output media, starting from a common master.

Given the amount of recent discussions about HDR images and scene-referred editing in this forum, I have the impression that those topics are not completely un-related to digital photography.

Currently the target of our edits is typically either a printed copy, or a digital image for sRGB-like display devices. However, my feeling is that the recent advances in display technologies will soon create the need for more sophisticated output transforms. Should we get prepared for that?

My understanding of the ACES approach is that it involves a single “master” copy of the final result, which is then “adapted” to different output media (standard sRGB devices, “HDR” TVs, cinema displats, etc…) by suitable “transforms”, which are more complex than simple colorspace conversions.

Should we follow a similar approach, and introduce such transforms in our image processing software, so that we can target a wider range of output devices?

Or should we put effort in improving the existing FOSS image viewers, so that they become capable of applying such “output transforms” on the fly?

I’d really like to know what you all think about this…

1 Like
2

Clearly demand for scene-referred, HDR etc. will only increase. Perhaps a gathering of information would be a good start. Then maybe some kind of minimal reference implementation (I guess some already exists)? A goal could be to include in as many applications as possible and make it fairly easy to do so, whilst also considering how difficult it might make things for the end user.

3

It appears to me part of the film industry push to scene-referred workflows was to coordinate the efforts of multiple teams producing content that, when composited, looked like it was shot in a single camera. That’s what’s (to my perspective, anyway) behind the 0.18 middle gray reference, that and working the images (camera-captured or CGI) in the radiometric space.

I think the key attribute for us singular still photographers is the linear editing thing. I’ve seen it in my use of rawproc, editing an image for tone or color after a display gamma has been applied changes the colors, sometimes for good, sometimes for bad. One thing I’ve learned from my day job is that working measurements on their original terms is preferable to doing such after they’ve been transformed in some way. An early such transform in our vernacular is the demosaic operation; there, we turn specific measurements of “bandpassed” light into assertions of presented spectrum at a location. One of the channels was measured; the others were statistically asserted. And so go the transforms, moving further and further from the radiometrics of the scene.

The nerf-bat of understanding that Troy whacked us with was fundamentally about how the legacy of the CRT display has taken us down a discrepant path, one where we attempt to make our images nice after they’ve been rendered “presentable”. I think we need to figure out ways to work our images in their radiometric space, and save the transforms for presentation for last. I’ve been playing with such in rawproc, precisely because there I can take a “really raw” image and stack a set of tools on it in any order i want, and see what goodness or badness comes from it. Indeed, I’ve almost gotten to linear editing/display transform order in my regular batch processing to proofs; I just need to shake this opioid-like need for applying the 1.8 gamma TRC when I first convert to the working profile.

On thing I’d recommend is maybe saving the term ‘linear’ for the radiometric relationship. I think ICC profiles have received a bad rap because the term ‘linear’ has been used to describe “identity” or “null” TRCs, gamma=1.0. As far as I can tell, using Little CMS to convert my raw image from the assigned camera profile to Rec2020 with gamma=1.0 is only doing the gamut mapping; anything I do to facilitate scene-referred workflow is going to depend on this, for the time being…

Good thread…

3 Likes
4

Where/where is this bad rap? I have puzzled over recent references to the bad rap of linear gamma ICC profiles and so far haven’t been able to figure out what anyone might be referring to.

It’s not the CRT display that led us down the bad path, leastways not for ICC profile color management. The problem was that we started editing digital images back when computers were not capable of processing high bit depth images in real time, or even at all. So we edited 8-bit images, which need to be encoded using a more or less perceptually uniform TRC, even when using ICC profile color management. The only other alternative was severely posterized shadows.

It wasn’t ACES or OCIO that made linear gamma image editing starting with scene-referred files possible. It was fast processors and more RAM.

The benefits of editing linearly encoded images are entirely 100% independent of whether one uses OCIO or ICC profile color management, and people including myself were editing scene-referred/“scene linear” image files in linear gamma ICC profile color spaces before ACES or OCIO started being widely discussed on the internet as the movie industry increasingly switched over to digital.

Go look at this tutorial over on the Krita website:

Pay attention to these sentences:

  1. It is easier to keep saturated non-muddy colors in a linear space.
  2. The high bit depth makes it easier to get smoother color mixes.
  3. Filters are more powerful a give nicer results in this space. It is far more easy to get nice blurring and bokeh results.
  4. Simple Blending Modes like Multiply or Addition are suddenly black magic. This is because Scene-Linear is the closest you can get to the physical(as in, physics, not material) model of color where multiplying colors with one another is one of the main ways to calculate the effect of light.
  5. Combining painting with other image results such as photography and physically based rendering is much easier as they too work in such a type of colorspace. So you could use such images as reference with little qualms, or make textures that play nice with such a renderer.

Now what specific item or items in the above sentences does anyone think might be specific to using OCIO instead of ICC? Take it apart, sentence by sentence, claim by claim.

I think @Carmelo_DrRaw has asked some interesting questions, but an informed discussion requires a bit more than relying on various people’s interpretations of what Troy has said about this and that. At the very least, it would be good to be familiar with the ACES primer and the specific list included in the primer of the problems ACES was actually designed to solve.

Here’s my assessment of what ACES has to offer Elle Stone as a photographer:

  • I’m not making movies. I’m not combining input from multiple cameras. I’m not sending footage out to be processed by different companies and then incorporating the various outputs into a master project.

  • I don’t have any concerns for saving edits on one image to use on the next image. Some people do, I just don’t. I don’t even care if I can save the history of edits on any given image because the next time I edit that image, I’ll be doing something different.

  • I do have concerns about archiving information and I avoid clipping until final output for display. But that’s what raw files and unbounded channel data and wider gamut ICC profiles are for.

  • I do have concerns about prepping (soft proofing) an image for display on various devices. But I absolutely do not need anyone to give me a “one size fits all” LUT to get from a suitable large gamut RGB working space to a given printer or even to sRGB. I actually enjoy soft proofing.

What’s your assessment? What benefit to you as a photographer would an ACES or ACES-like workflow bring?

Please note the above question of “What benefits would an ACES workflow bring to your workflow” is quite apart from any specific editing algorithm such as CDL-style color correction or filmic tone-mapping. Which by the way the PhotoFlow filmic module just rocks, and it was coded directly from the original filmic equations which contrary to what some people might inadvertently assume weren’t written by Troy. Troy wrote an implementation just as @Carmelo_DrRaw did, but the PhotoFlow filmic implementation is better imho.

1 Like
5

I watched the previous discourse go awry at that very term, for the reason I described. We think of a 1.0 TRC as “linear” because the transfer function plot (curve, if you will), is a straight line. Math-wise, it should be referred to as “identity”, where the application of it makes no change to the input. Then, we can use “linear” for the relationship between increasing light intensities, where math-wise it does make sense.

Disclaimer: I Am Not A Math Person.

1 Like
6

Do you have a link?

Putting the question of whether the color management system is ICC or OCIO to one side, there are benefits to converting an image to a linear gamma ICC profile, or making the encoding linear, linearizing the RGB, call it whatever you want, if you plan to do further editing. One such benefit is downsizing without gamma artifacts. Another is blurring without gamma artifacts. Another is being able to use multiply to give a color cast, without gamma artifacts. And if you want to composite two layers together, and on and on and on. These benefits are entirely independent of whether the starting image was actually scene-referred or not. Maybe it’s already undergone some “make it pretty” edits.

There’s only so much you can do to an originally scene-referred image using only edits that preserve scene ratios.

If one is painting a picture, there aren’t any scene-referred light intensities. And yet it still makes sense to paint using linear gamma ICC/linearly encoded RGB/linear light/I don’t care what you call it as long as it’s linear.

Hmm, I don’t think the distinction you are making is useful. We have “scene-referred” to refer to channel values that are equal or proportional to scene values. Why do we also need to separate out “linear gamma ICC” as one way to encode data, from linearly encoded channel values using some other form of color management, that might or might not also be scene-referred?

What is the point of saying “linear” must refer to “scene-referred”? How then do we talk about linear encoding of RGB with all the editing benefits thereof?

What does “no change to the input” mean? Please don’t assume that I don’t know what “identity” means. An ICC profile just specifies how to interpret the RGB channel values. It doesn’t do anything else at all. And the TRC, white point, and primaries (assuming an RGB matrix profile) are all critically important for specifying which color in XYZ goes with which trio of RGB channel values. So again, I just don’t know what you might mean by “no change to the input” - what is the “input” that isn’t changing, and how should “it” be changing if the TRC isn’t linear?

7

The issues of terminology and communication have to do with your background, profession and personality. This applies to life in general. When people talk past each other, it is often because they haven’t defined their terms or had the charity to understand the other.

Personally, I think linearity is important because it is mathematically simple. Take for example @Carmelo_DrRaw’s exploration of the colour module and log2(). Linear is much simpler to understand and also faster to compute. Image processing papers, at least the groundbreaking ones, tend to use linear images and algorithms as their foundation.

The further you abstract, the more difficulty you would likely face. However, that is just a general statement. There are instances where having a non-linear intermediary can actually be beneficial. It is up to the programmer and / or the artist to weight the pros and cons.

PS The above is just about linearity and not about identity or scene-referred. The main things to consider are: Linear relative to what? What is identity? Even scene-referred isn’t as pure as one would think. That is because we don’t or can’t capture ground truth. Raw data needs to be interpreted or at least transformed into a format that we can use. Take a course on scientific instrumentation (or something like that in another discipline) and you would begin to appreciate how complex it could be.

2 Likes
8

I’ll find an example, but it will take a bit. I just remember reading an exchange between Troy and someone else I can’t recall, and the disconnect stuck out like a sore thumb.

Fair enough. Identity (mathematics) - Wikipedia

For instance, 1 is a multiplicative identity. Multiply any number by 1, you get the same number. With respect to TRCs, “identity” would be a transfer function that yields the same image with respect to tone.

That’s how I use your g10 ICCs to only do the colorspace part of the conversion. Let me know if I’m missing something there.

Okay, I just tried something, I had a “linear” image open, so I put in a colorspace tool, converted it to sRGB g2.2, then put in another colorspace tool, converted it to sRGB g1.0. The histogram spread out for the first conversion, then went back to its orginal shape with the second. Okay, so calling g1.0 “identity” isn’t correct. What it did in the above case is to restore the linear relationship of the tones, based on knowing that they were 2.2 in the input. I stand schooled…

2 Likes
9

It would compel me to hold the image data to its original radiometric relationship as long as possible in editing prior to transforming it for display/print. It would fall into the same category as does maintaining “bit depth” as long as possible.

1 Like
10

Hmm, actually I really do know what “identity” means, my apologies for not speaking more clearly.

Take an image with an assigned ICC profile. That profile correlates every trio of RGB channel value in the image with a corresponding trio of XYZ channel values. The TRC is a necessary part of the specification. I actually don’t know what “same image with respect to tone” might mean at all.

1 Like
11

You shouldn’t. I got it wrong.

12

Yes - converting a scene-referred image to a nonlinear RGB color space doesn’t preserve the original scene ratios - colors are the same, ratios in the encoding are not. But as soon as you convert back to linear RGB, colors are still the same and ratios are restored, assuming of course all that was done was a lossless conversion from a linear encoding to non-linear and back to linear again.

Edit: our posts crossed :slight_smile:

1 Like
13

Linearity in image processing can appear at 2 tricky levels:

  • In 1D, it’s a transfer function of the general form f(x) = a * x + b. A such function does NOT keep the original ratios, since f(x*n) = a * x * n + b ≠ n * f(x).
    n * f(x) = f(x * n) if and only if b = 0, so the ratios are preserved only when there is no lift (b factor). Identity is a particular case of linear 1D function where a = 1, hence f(x) = x.
  • In 2D and 3D, it’s a linear map defined as the transformation between 2 vectorial spaces that holds the properties of homogeneity and additivity. These are spatial transformations, such as rotations, scaling, projections, etc.

Said simply, a linear operation is a an operation that can be applied twice on the image and have the same effect than applying once with parameters twice as large. Ex: a rotation of 90° is the same as 2 rotations of 45°, raising the exposure by 1 EV twice is the same as raising the exposure once by 2 EV.

As said above, this triggers interesting properties for blending modes, rotations, blurs and edges preservation.

ICC are utterly bad because they use transforms such as f(x) = x^(1/gamma) which don’t keep the ratios. You might want to use them as outputs, never as inputs. CRT screens use to have a gamma 2.2 optical tone response, so one would have to reverse that TRC numerically to get a proper grey scale.

But where ICCs have hurt big time is they use gamma inside color spaces too. This is not anymore to revert the CRT optical behaviour, but just to give more code values to the darkest stops when encoding in 8-16 bits unsigned integers, and avoiding posterization and quantization effects. And yet it’s called the same, and confuse everyone. These TRC are completely optionnal and not needed at all as long as the bit-depth is enough.

And what is even worse is the actual implementation of ICCs in free softwares. For example, when you have an color-managed app that fetches the OS ICC profile, on a color-managed OS, you have no idea if what you see is sRGB projected in display RGB or display RGB projected in itself (hello #darktable !), especially when the pipe is actually Lab. It seems that, as long as you put ICC on top of your crap, you feel safe because suddenly, color is managed somehow.

1 Like
14

hi,

can you please elaborate a little bit? are you talking about Mac OS perhaps?

15

All I know about Mac OS is I don’t want to use it.

But on Linux + Gnome / Budgie, when I change the screen color profile with darktable running, it changes the color in the darkroom. So the OS applies color correction inside darktable. But darktable applies a color correction in itself by fetching the OS color profile through colord or xatom.

So, what am I staring at inside darktable ? A double correction or a more accurate one ? I have no idea, and that is wrong in itself.

16

You do not need ACES for that… this can be achieved in any processing pipeline, as long as the data is kept in linear RGB. Moreover, any non-linear adjustment (including the recently discussed ASC-CDL color corrections) will break the radiometric relationship. Yet, such adjustments are better applied to linear RGB data, and should return linear RGB data.

Nowhere in the ICC specifications it is written that they must use a gamma-like TRC. My understanding is that the gamma-like TRC have been introduced as part of the standard in order to address both the non-linearity of CRTs and the need to optimize the bit usage in 8-bit processing. However, nothing prevents to create and use an ICC profile with sRGB primaries and a linear TRC, and actually @Elle is building and shipping such type of profile in her collection.

My feeling is that people are taking all those standards (ICC, ACES, etc…) as philosophical truth, while instead they are simply tools that provide a standard solution to specific technical problems. You do not need ICCs or a CMS to go from ACES to a display profile, you can derive and apply the math yourself. But ICCs and a CMS are there to simplify your life…

I think the approach should be to provide a well-established standard way of proceed when transforming the scene-referred linear data to output-referred format, but also provide enough flexibility so that “expert users” can go their own path without being limited by the choices of the software developers.

4 Likes
17
  • IDTs – While major players like Arri, Sony, Canon and BMD have IDTs for their camera systems, there aren’t specific ones for every camera. Sure, there are generic Rec 709 or P3 IDTs, but in my mind, the generic nature of those IDTs in some ways defeats the precise scene linear approach of ACES. Of course, as new IDTs are developed by camera manufacturers the options expand. Are you working on a project with consumer cameras that don’t have specific IDTs? ACES is probably not the right workflow for you.

https://mixinglight.com/color-tutorial/getting-know-aces/

18

I guess that if one starts from a RAW file and uses a “neutral” camera profile - like one obtained from an IT8 taget, without any “look” backed in - one can assume that the input data is “linear scene referred”.

My impression is that such an IDT would be for example needed to obtain “linear scene referred” values from an in-camera Jpeg, but I would not go down such rabbit hole in the digital photography case. After all, the RAW files are exactly meant for this…

19

For reference, this can easily be done using RawTherapee’s built-in ICC Profile Creator.

3 Likes
20

From my consideration of the past discussion, I think that “scene referred” would also include an exposure compensation adjustment to put middle gray at 0.18, in a 0.0 black-anchored data space.

So, at this point, I think I’d recommend considering “linear” and “scene referred” distinctly. In the ACES world, the second appears to include the first. But for individual endeavors, “linear” is the more important thing to consider, “scene referred” is more about coordinating the efforts of many.

Those that are more familiar with the film production juggernaut are welcome to skewer this thinking; well, you’re all welcome to skewer, I’m just a goofball trying to figure it out…

1 Like