Processing RAWs for HDR displays in Darktable

Thinking a bit more about it, wouldn’t the Tone Equalizer module be the perfect tool for generating such a gain map? (Or, in this case, a tone map, since this would correspond to Example Use Case #3 in the spec, i.e. a base HDR image, and then a gain map to tonemap it down to SDR.)

The base HDR image (with unbounded luminance) would be obtained by disabling Tone Equalizer, while the gain map could probably be computed from the exposure mask.

2 Likes

See also Ultra HDR Image Format v1.0  |  Android Developers

2 Likes

Looks interesting! Do you know who is developing it?

I hope this isn’t yet another Google technology that will end up in the Google graveyard.

EDIT: Seems like it is. Some relevant PetaPixel article. The promo picture looks like a good ol’ overprocessed HDR :grimacing:, but this might not be the fault of the technology itself.

It’s a Google implementation of the Adobe Gain Map. Basically straight out of the specification, but with a Google GContainer thrown in for good measure. Not sure why they felt the need for that, since it doesn’t seem to do anything that MPF doesn’t already do well enough.

1 Like

Triggered by some articles about HDR workflows, I started to take a look at DT capabilities to prepare images for display on HDR displays. Because of the unbound data in the scene-referered workflow I thought, that shouldn’t be a big issue at all and started to read a several threads here.

For testing, I took an image with a large dynamic range and increased exposure so that the brightest non-overexposed pixels are above 255 (using the pipette in RGB mode) without applying tone mapping. Then I played with sigmoids “target white” value and increased it to 400% so that pixels with highest intensity got close to a value 1024. This is what I somehow expected.

Then I exported the image as 10bit avif / jpeg xl using the ‘linear Rec2020 RGB’ profile … well I think I need to read a bit about the different formats and profiles and their compatibility.

The question that I have now, is about the histogram in Darktable 4.6.0. It changes upon changing the histogram profile, but to me it looks like, it is always restricted in values on the x-axis. When I increase exposure, it will not show the entire histogram. But I wonder why, because the work profile is Linear Rec 2020 and values should go from 0 to 1. Is this a limit of the histogram chart or do I make something fundamentally wrong? Is it possible to show the entire histrogram?

I attached a screen shot as well as the raw file with xmp.

Best regards and a happy new year!
Till

P.S. I don’t have a HDR display on my laptop, but consider getting an external one, so I can only judge the results on the phone/tablet.

20230226_165438-_MG_2170.cr3 (25.3 MB)
20230226_165438-_MG_2170.cr3.xmp (30.0 KB)

This actually causes more problems than it solves, because the output formats are not unbounded unless you’re exporting JXR - which I don’t think darktable can do.

I still can’t understand why darktable users are so afraid of the exposure compensation slider… There’s this unhealthy obsession with “unbounded” when the reality is that any output format IS bounded except for floating-point TIFF which is really only usable for feeding to another piece of software for further processing.

Back when I did use darktable, I did the following:

  • Adjust exposure compensation so that the highlights are just under clipping on the histogram (shouldn’t need to be much, since fundamentally the input device is bounded too. Highlight reconstruction and colorspace conversion could result in some positive gain that causes clipping but generally not muh.)
  • Disable all tonemapping modules (filmic, etc - sigmoid didn’t exist back then) - they’re not necessary for an HDR display
  • Export as linear Rec. 2020 16-bit TIFF
  • Convert to 10-bit Rec. 2020 HLG H.265 video using Experimenting with HLG and HDR10 for still images - #22 by Entropy512
1 Like

Actually, any floating point output from dt is unbounded (EXR, JXL, PFM, TIFF).

1 Like

I will take a look at the script, to convert DT‘s output to HDR files that are compatible with HDR screens. Thanks for pointing to the discussion!

I am using the exposure slider a lot, but still don’t understand the histogram in DT beyond 8bits/sRGB images. If I increase exposure, the histogram moves to the right and finally it moves out of the shown range. Is the right axis limit set to 1.0 and any value above is not shown?

Well, exporting images as 32bit and plotting the histogram with some Python script will reveal the answer…

Did you change (to which software) because of HDR related issues?

BR
Till

Indeed, the histogram currently does not show above 1.0, but not of sRGB, but your display color space. It’s all in the manual.

If you have an idea how to improve this for HDR workflow, feel free to open a feature or pull request on GitHub.

Oops I missed that part in the manual. I think first I need a HDR display for my laptop to see how this will change the situation.

I don’t know for you, but for me, converting to linear Rec. 2020 16-bit TIFF is not satisfactory, as it will clip the colorspace of my camera, which is a bit larger than ProPhoto RGB. In other words, clipping colors is unacceptable to me, so here is a linear Rec2020 profile with luts instead of a simple matrix. Use it with the perceptual rendering intent. The gamut mapping is done from ProPhoto RGB to Rec2020.

However good that solution may be, it’s highly unlikely that the display at the end of the line will show anything close to the Rec2020 colorspace, more clipping in sight. And from what I’ve seen so far, HDR in the real world is a monumental mess.

My objective for going the HDR route, is mainly a color gamut thing and a little bit for higher dynamic range. But it looks like all this maybe futile.

I’m in search of a solution to bring my (very) wide gamut work to the smartphone world, especially.

Any suggestion?

LUTRec2020-elle-V2-g10.icm (1.3 MB)

I am just wondering and your comments on color made me think of the days of my youth when people were buying HIFI sound systems and speakers capable of generating a range of frequencies outside the range of human hearing.

If what we see in the world around us is essentially the reflected light from the surfaces around us and this is or has been quantified at least by one researcher as the so called pointer’s gamut…then why do we need to worry so much about colors that in some cases are so far outside this gamut…

I’m sure there is some nuance to this… I guess it might lie between reflected light and direct light and what we can perceive?? Any way I am sure its one of those dumb questions I should just think on more…

2 Likes

I don’t know much about the Pointers gamut, but here is a quote from the same article.

If you interpret this as I do, the CIE 1931 xy chromaticity diagram is essentially all the colors a human can see and the Pointer’s gamut is a subset of that. At the moment, no device we have created comes close to what we can see, but modern camera have the widest gamut of all these devices.

I often hear, but we can’t see these colors, way bother? All those who say that, are still living in the 90s when all display were cathodic tube, Today, some displays can achieve 95% of the Rec. 2020 color gamut, yes sir, that’s color. (Christie CP4415-RGB).

I know no one of us will ever have these 50K device in their living room. Remember, plasma display were in 10-20K range not so long ago, now, they’re basically obsolete. I just decided to go HDR just so people can see my flowers in all their “glorious” color spectrum.

It’s people like us that will pressure manufacturers to push the limit. To me, a modern DSLR camera or mirror less one is like a Ferrari, but some folks would tell us to run it on only 8 cylinders (out of 12) to save fuel or whatever, my god, what a shame. I would be among those who say, give roads where I can go 300 km/h?

I just found a device that will show me an acceptable HDR10 (1000 nits, 100% DCI-P3) image for the money I can afford, it’s only to show others since I’ve chosen a long time ago to have wide gamut displays (99%+ Adobe RGB). For all my personal stuff, I develop my raw files using either dcraw (LibRaw) or I use a custom flat DCP profile in Lightroom. I don’t do real work on anything less than a 16 bit tiff file with ProPhoto RGB colorspace. For critical printing (which I do myself), I do what is called image dependent gamut mapping. Some also call this late color mapping.

You’re right, I’m a color maniac, as of know I tried my best to deliver at least with my prints the widest color gamut I could realize with the available material out there. Now, smartphones are the way to go if you want to reach the masses with an explosion of light and colors. I think I’ll hav e a lot of fun…

~Yves

1 Like

I find smartphones very underwhelming for any kind of image viewing because of their size. What I’m interested in are the ultrawide monitors of at least 34" like the Alienware AW3423DWF. I still think I’ll wait a couple of years though to see what new options are available by then. OLED still has some issues.

Ya it might be a wait and see time…there are new pholed displays coming in the not so distant future it would seem…

Yep, it will be interesting to see what PHOLED can do. I think the holy grail is microLEDs, but it will be a few years before they are widely available and affordable. And that will only be TVs first. Monitors tend to receive the new tech a lot later.

As TV’s have cropped up in the discussion, anyone know how colour accurate TVs are usually? I suppose they can be profiled just like anything else, but when I walk through a consumer electronics store I’m dazzled by the array of different hues on a bank of different models, and all looking horrendously over saturated. Just yuck. :crazy_face:
But then it’s not exactly a good viewing enviroment either. :face_with_raised_eyebrow:

I don’t have a TV so…

Generally not great, they’re engineered to look good, not be accurate.

1 Like

They do that on purpose. The thinking being (not unreasonably) that most consumers associate saturated colors with a better TV, but also to grab their attention in the shop.

1 Like

Though I agree with you, and we won’t beat a large (wide) screen for the immersion feeling it provides with a smartphone any time soon. But the reality is, the market for “HDR TV” is shrinking like snow in the sun, so I don’t want to target such displays.

My setup for viewing my photos, is essentially wide gamut SDR, and when I want people to see my work as I created it, my photos end up being viewed, 99%+ of the time, as sRGB’s on SDR monitors. I find this unacceptable, and I’m looking at my options so that at minimum my photos would be seen as wide gamut images. Ideally, that would be on a Rec2020 display, which would still squash my color by the way. But who as the kind of money needed for such capable device.

~Yves