Unbounded Floating Point Pipelines

Processing
244

Okay I investigated Natron a bit further and the biggest issue is that the core application is extremely minimal[1,2], so although it sets everything up for the plugins to be able to use OCIO configs it doesn’t really uses it internally, which becomes a problem with the view node[3] since that is not provided by a plugin but by the core app. Since the view node doesn’t use OCIO but something internal, which is a shame since you lose a lot of the advantages of OCIO that way.
Now I did found a bit of a workaround but it is a bit of a kludge, using an OCIODisplay node and putting the view in linear, I can then use the controls of the OCIODisplay node to apply the correct view settings.

Natron_OCIO
Natron_OCIO1960×1213 236 KB

As I said this is a bit of a kludge so I am looking into how to get the view node to support the OCIO config, afterwards it would also be best to curate the plugins so the default set is scene referred only or at least make it more clear which ones should only be used on display referred data.

[1] This is caused due to (if I remember correctly) Natron was originally developed as a testbed/development environment for OVFX plugins so they originally only implemented enough to do that, of course it has grown a bit since then
[2] There are some other issues mostly in that the merge node includes some operations that shouldn’t be used in a scene linear workflow and the default distribution includes some plugins that shouldn’t be used on scene referred data (like an inverse node)
[3] The read and write nodes do require a plugin to be functional and those plugins can (and do) use OCIO, so the only node we care about is the view node

1 Like
245

Yes, that’s the reason why Natron wasn’t really recommended as an example of implementation, but good work at spotting what’s wrong with it, it shows you’re on the right track.

246

Although this work around makes it rather kludgy to use in a real workflow[1] I do think it is workable for our current discussion and examples although it does require a bit more care and attention then other programs (since the default install comes with display referred nodes that are improperly labeled), have some ideas on how to fix this and although I am not a programmer I don’t think it should be to hard (the OCIO API seems to be pretty workable)

[1] Although apparently people do manage to get some rather impressive results although I suspect this is because the Natron internal defined colorspaces match those of blender so the results are “good enough”

247

Just for curiosity how to get scene referred from a still raw? It look non trivial to have
channel values proportionate to the scene as captured by the sensor and make it compatibile with aces.

“CameraRAW RGB data (generated by libraw) is converted to ACES by calculating an Input Device Transform (IDT) based on the camera’s sensitivity and a light source”
https://github.com/ampas/rawtoaces

https://www.magiclantern.fm/forum/index.php?topic=10626.0

Maybe Spectral Sensitivity data are needed?
http://acescentral.com/t/rawtoaces-calling-all-developer-types/1048/9

248

@age - the documents you reference are about shooting video rather than still images.

With a still image use an input profile for your camera. I’ll defer to whoever wants to answer which camera input profile you should use, whether an ICC profile or an Adobe Dng profile.

I use a general purpose custom camera input linear gamma ICC matrix profile made using ArgyllCMS, but whether that’s appropriate for use in an OCIO workflow, I don’t know.

I don’t know what “scene-referred” and “scene-referred workflows” actually means in the context of this discussion of OCIO workflows. I asked @gez but didn’t get any answer.

Here is a nice presentation of one person’s understanding of “scene-referred” in the context of ICC profile color management:

http://simon.tindemans.eu/essays/scenereferredworkflow

I’ve heard two different definitions of “scene-referred data” - not meaning the whole workflow, just the starting data as captured by the camera:

  1. Matching as closely as possible the intensities in the original scene.

  2. Proportionate to the intensities in the original scene, which would allow any editing operation that preserves proportionality, such as setting/changing a white balance, adding positive/negative exposure compensation.

My understanding is that operations such as removing lens distortion, chromatic aberration, and such are part of make a scene-referred file.

My further understanding is that there is no such thing as “100% completely scene-referred data” but only “sensor-referred data” - you are limited to the data your camera actually captured. If anyone on this list has a way to reach past the sensor data to the actual scene data, I’m curious to hear how this happens. Ultra-expensive cameras in research labs don’t exactly count in the context of the current discussion. I’ve heard there is a camera that actually fulfills the Luther-Ives condition, but again that’s not very relevant to the cameras most/all of us are using right now.

In my own view, “scene-referred data” just means “as scene-referred as possible given the camera, lens, raw processor, and input profile, plus the actual scene lighting, as general purpose input profiles aren’t as accurate as profiles custom-made for each lighting condition, and etc.”

And then you have to take into account camera settings, as some camera settings aimed at compressing “higher dynamic range than camera can capture” or “bringing out shadow detail” by design alter the recorded intensities.

And then there are considerations of the linearity of the sensor’s response, which of course breaks down in the shadows and also near the full well point. Plus there’s whatever processing the camera firmware does to the data before saving it to disk, which might further compromise the integrity of the light intensities that were actually captured by the sensor.

1 Like
249

Probably some more step are involved :thinking:
http://acescentral.com/t/rawtoaces-calling-all-developer-types/1048/4

The software uses libraw to open a wide range of RAW file formats. It then uses some metadata in the files to try to figure out the source used to photograph the scene. If the spectral sensitivities of the camera are known they are used to derive an IDT based on those spectral sensitivities and the “adopted” scene source. That IDT is then applied to the image and the results are placed in an SMPTE 2065-4 compliant ACES container (exr) file12.
If spectral sensitivities of the file aren’t available, the software uses metadata in the file to do our best to convert the files to ACES, but your mileage may vary here

From what I understand all raw are in the 0-1 range and the IDT is adopted because for example one camera with 8 stop of dynamic range is mapped in the 0-1 range and a camera with 15-stop is mapped in 0-15(random values) range.

250

Yes, this is one of the things that puzzles me. Some scenes only have a small dynamic range. So how does this work in an ACES or other OCIO workflow?

If the camera can capture say 12 stops, and the scene itself only has 8 stops, is exposure compensation - which is what I think is meant by “mapping” - applied to expand the 8 stops to fill 12 stops?

Here is a possibly related question, which has to do with middle gray: When speaking strictly in terms of processing a raw file to make a nice photograph, where you put middle gray in a scene is dictated by the photographer’s artistic intention. And the exposure used to capture the scene is likely more related to avoiding clipping than to setting the exposure to make the final middle gray actually measure 0.18 in the captured raw file.

Though of course you can make bracketed shots and merge them together, which still leaves the question of "how is middle gray determined in an ACES and/or other OCIO workflow?

In a scene with say 15 stops of dynamic range as measured from the darkest shadow to the brightest specular? diffuse white? areas in the scene, how does an ACES or other OCIO workflow deal with where to put middle gray? Is this somehow contained in the captured data? Or is it a decision made by the photographer.

251

Scene-referred means mantaining the energy-based reference.
Or in other words, keeping the energy ratios from the scene in the captured image.
If that piece of paper under the sun is 2 stops over middle gray in your scene, it will be 2 stops over middle gray in your capture, and so on with every spot you sample.
It also means that the energy can go from 0 to infinity technically, although in the case of cameras you’ll be able to only capture a portion of the scene’s dynamic range in a single shot.

Scene-referred workflow is a workflow that aims to keep those ratios in the actual pixels. You can produce display referred outputs from the scene-referred image, you can also stack creative modifications, but the underlaying pixels are always in reference.

One of the benefits of a scene-referred workflow is that you produce files that keep the scene’s light ratios. This is extremely useful for compositing and CG in general.
Display referred workflows only produce imagery with the output characteristics baked in. That’s ok for final delivery but very limiting for compositing and further editing.

252

OK, where do you put the piece of paper in the scene? That will make a difference in what middle gray is.

253

What? Paper has a reflectance of around 70% which means it will bounce around two stops above a gray card reference (18%).
Expose your image as you wish, it will be always 2 stops up if both are under the same lighting.
Discussing this sounds as discussing the very basics of photography, but maybe i just missed what you mean.

254

Yes, I know that. Though I would say perhaps more like 85% for white office paper. But maybe I’m wrong.

I think maybe you missed what I mean. Consider photographing a beach scene, bright sunlight, bright water, bright sand, a white building in full sun with deep shadows on one side, a nice large umbrella on the sand casting a lighter shadow, with people under the umbrella, more people in the shade of the building, and some strollers along the beach with their dogs.

Where do you put the white paper? I’m guessing you will say in the full sun. But middle gray in the final image depends on the artist’s intentions.

So let’s try another scene holding a store window with an interior scene and lighting, and some space alongside the building with outdoor lighting which might be brighter (on a sunny day) or dimmer (a very overcast day or after dark or etc).

Where do you put the white paper? Always in the brightest light?

What if the scene is overall rather dim with multiple light sources, but a small part of the scene off to one side has a very bright light that doesn’t cast illumination very far. Where do you put the white paper?

I think maybe you will say the white paper always gets put in the brightest light in the scene? If so, how does this work with a scene with specular highlights?

255

In my opinion there are two definition of what are scene referred

  1. the one described by Elle
  2. the “real” scene referred takes into account the specif dynamic range of a sensor, spectral sensitivities data and generally some kind of meausurements that nowday are available only for high end cinema camera.
256

Engineer here, I think the 0.18 gray reference is very useful in a workflow where the image capture-ers are different folks than the image-worker-uponers. The capturers make a determination where gray sits in the scene, and deliver data that is gray-referenced to that location. Then, the workers use the data set accordingly, knowing where gray is intended to be placed in the eventual output. FWIW…?

2 Likes
257

As someone who knows absolutely nothing about this topic I thought I’d butt in :slight_smile:

You seem to be emphasising this intermediate (exr) file and producing output which is scene referred. So its basically about compatibility wrongword and workflows spanning multiple software. How critical is this for normal photography? Usually once you’re outside the raw editor you’re doing tweaks for final delivery anyway.

258

Hmm, when shooting digital raw files, normally one doesn’t expose for middle gray as doing so runs the risk of clipping highlights, and once the highlights are clipped that’s lost data. When shooting jpeg, that’s a different story. And maybe shooting movies is yet different again, but I don’t know anything at all about video.

So capturing a still image shooting raw, the important thing is to avoid clipped highlights (well, maybe those highlights are so small and unimportant as to not matter at all, that’s a judgement call), and then setting middle gray is done somewhere else in the workflow pipeline.

259

Well, white paper is a horrible reference. That’s why photographers use a gray card instead.
I just mentioned white paper as an example to illustrate that the ratio is kept. Discussing where to put the paper or gray card in the scene sounds pointless.
What’s important here is whatever you captured in your photo must retain the energy ratios of the scene.

260

Most cameras are integer based, and linearly encoded. This means that despite all values being normalized to 0.0 to 1.0 range, they can represent varying dynamic ranges, depending on the encoded value positions.

I believe there is, again, conflation going on.

Creatively, one can put an ostrich in the scene. It doesn’t matter. Technically, code values require a ground truth to gain meaning.

Regarding workflows, one requires known code values to map to scene referred code values. Hence, middle grey is important not so much because of any perceptual element, but because it forms an anchoring point. Same is required for a view transform.

In a theoretically plausible scenario, a perfect white diffuser will reflect 100% of the incoming light. That is, if an 18% reflectance were lit with a singular source, a perfect white diffuser would yield a code value of 1.0. Note that we can’t work backwards and suggest that a code value of 1.0 represents a perfectly white diffuser.

1 Like
261

I think that my first definition “Matching as closely as possible the intensities in the original scene” is perhaps the same as @gez 's “Scene-referred means mantaining the energy-based reference.” But maybe I’m wrong, and maybe this is another case of terminology mismatch. But the actual meaning of “maintaining the energy-based reference” is not obvious, at least not to me. I’m having to guess the meaning from the context.

Assuming I do understand what “maintaining the energy-based reference” means, in other words assuming @gez means matching the intensities in the original scene, the problem is how to scale what the camera captured to match the actual intensities in the scene.

262

That’s the problem

264

Is the value of “STOP_ADJUSTMENT” equal to dynamic range of camera, say 12 stops, minus the 8 stops that presumably fit within 0-1? Which would be “4” in this example? Or is it equal to the dynamic range of the camera, in which case in this example “12”?