Processing RAWs for HDR displays in Darktable

Software darktable
,
32

Looking at this comment again, it seems that intensity_target denotes the maximum theoretical luminance of the curve (i.e. 10000 nits for PQ, even if the image is limited to 1000 nits), so that multiplying it by Y gives the displayed luminance.

If the output color profile specifies the screen luminance (e.g. 80 nits for sRGB), it would make sense to use this value. But better double check that with Jon or someone who knows the JXL standard well, to make sure that decoding SDR images does not rely on the special value of 255.

33

I just noticed that Google Chrome is already capable of displaying HDR images in the JPEG XL format.

See my comment in the JXL PR discussion for more details.

The trick is to enable the flag chrome://flags/#enable-jxl and, using the above-mentioned DT patch, to export the images in JPEG XL format with the PQ Rec2020 RGB profile. The images are a bit slow to load in Chrome, and lossless JXL files never load, but apart from that it works, with accurate colors and contrast. HLG looks okay too, but the shadows are a bit too contrasty, while the linear transfer curve leads to posterization and JPEG artifacts in the shadows.

A 24 MP picture now weights only 4.5 MB in JXL format (for quality=12) vs. nearly 300 MB in EXR format. And it looks even better since the EXR was not color managed properly.

When I find time, I will try to tinker with JXLook and see if I can get it to display the HDR luminances instead of clipping them.

2 Likes
34

Simple steps to PQ encoding with darktable and ffmpeg (x265 output)

P1000657.RW2 (18.8 MB)
P1000657.RW2.xmp (20.7 KB)

  1. load the image

    hdr-o
    hdr-o1644×1032 313 KB

  2. add exposure compensation to taste

    hdr-exposure
    hdr-exposure1644×1032 388 KB

  3. map 10000nits to 0-1 range (exposure setting -6.42)

    hdr-exposure2
    hdr-exposure21644×1032 182 KB

  4. export as rgb pq rec2020 tiff

5)encode with ffmpeg, in this step we only convert to ycbcr and attach the metadata

ffmpeg -loop 1 -t 10 -i P1000657.tif -vf scale=out_color_matrix=bt2020nc,format=yuv420p10 -c:v libx265 -preset medium -x265-params “colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:range=limited:master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1):max-cll=1000,0.0001” out1.mp4

Simple steps to HLG encoding with darktable and ffmpeg
It can’t be done right now but the steps for hlg encoding are the follows
https://www.bbc.co.uk/rd/sites/50335ff370b5c262af000004/assets/592eea8006d63e5e5200f90d/BBC_HDRTV_PQ_HLG_Transcode_v2.pdf

  1. load the raw and add exposure correction

  2. map 1000 nits to range 0-1 ( pixels divided by 10 or exposure corretion tool -3.321 ev)

  3. apply system gamma (γ=1.2+0.42Log10 (lw/1000)=1.2+0.42Log10 (1000/1000)=1.2)

  4. eventually expand the 0-1 range to 0-12 range (based on what formula darktable use)

  5. convert from “linear” to hlg

2 Likes
35

Amazing!!!

I followed your instructions, and this time it worked perfectly! I managed to obtain a MP4 file which correctly renders as HDR in both QuickTime and Infuse, and for the latter it seems to match the JXL file quite well (when assuming that the SDR brightness is about 64 nits, not sure why…). See the comparison below:

IMG_3162_annotated
IMG_3162_annotated1920×1373 202 KB

(Unfortunately, not many open-source viewers seem to support HDR :flushed:)

It also renders consistently on my iPhone XS (after adjusting the brightness), but obviously this one isn’t in the picture.

This time I have used the base curve (since Aurélien said it is not bad any more :stuck_out_tongue_winking_eye:) to flatten the highlights a bit, and I have set the max luminance to about 1000 nits (by adjusting the histogram after base curve to the right, then adding an exposure correction of log2(1000/64)=+3.97 for JXL, log2(1000/10000)=-3.32 for H.265).

I have also tried to use filmic and then correct the exposure, but I could not get a good looking result.

Btw, it is interesting to see that depending on which method we use, “luminance=1.0” (i.e. histogram to the right) is mapped to either the SDR luminance (when exporting to JXL) or to the HDR one (when exporting to TIFF and using ffmpeg to encode it as H.265).

36

Below are all the sample images from my original post, in JPEG XL format (quality=14, i.e. visually lossless) and full resolution. I have used the PQ curve since it gives perfectly accurate results in Chrome.

samples_jxl.zip (33.9 MB)

I had to zip them because the forum software does not recognize the .jxl extension yet and won’t allow me to upload them one by one.

Sorry, the file you are trying to upload is not authorized (authorized extensions: jpg, jpeg, png, gif, ico, dtstyle, txt, scm, pp3, svg, xmp, bz2, xcfbz2, py, arw, apng, orf, cr2, nef, dng, tif, patch, zip, 7z, rtc, raw, exr, hdr, raf, rw2, pfi, xcf, xcf, pef, icc, icm, pto, blend, dcp, xcfxz, xcfxz, ods, mp4, mkv, ogv, webm, cr2, nef, dng, cr2, pdf, kra, ntp, arp, fits, seq, gz, ssf, cr3, heif, heic, avif, crw, dtpreset, nrw).
1 Like
37

The future of HDR images on the web will be “gain maps”. This allows an image to look optimal both on SDR and HDR displays (as tone mapped HDR looks pretty mediocre on an SDR display, and sharing 2 versions of an image online is cumbersome, especially given mixed support for media queries and the tag).

Adobe has released a spec and sample images in JPG, AVIF, and JXL formats: Gain Map in Adobe Camera Raw

Chrome Canary already supports JPG gain maps under a dev flag (they refer to the format as Ultra HDR JPG).

1 Like
38

Microsoft is making some changes as well to try and manage all this for the OS and apps… Advancing the State of Color Management in Windows - DirectX Developer Blog

1 Like
39

Thanks a lot for the link! This is a really interesting technology. Let’s hope that OS/browser support doesn’t take as long to arrive as for “HDR only” JXL/AVIF.

Adobe’s gain map demo app might actually be the best viewer so far for HDR JXL/AVIF images on macOS Ventura and earlier (it even supports slideshows!). I will probably use it as a reference for reporting rendering bugs, and to check whether some HDR image exported from Darktable renders badly because of the viewer or DT (or my export settings…). So far I was using my iPad for this purpose (using the iOS 17 developer beta), as an “external” viewer, and it was always difficult to know which side was at fault when some image didn’t render as expected.

I’ll be experimenting more with HDR+Darktable in the coming weeks, since I want to re-process some of my photos from last year(s) to HDR (not targeting the web for now, just slideshows on iPad or TV). I’ll try to post an update at some point, but it might take a while due to limited free time.

1 Like
40

Thinking a bit more about it, wouldn’t the Tone Equalizer module be the perfect tool for generating such a gain map? (Or, in this case, a tone map, since this would correspond to Example Use Case #3 in the spec, i.e. a base HDR image, and then a gain map to tonemap it down to SDR.)

The base HDR image (with unbounded luminance) would be obtained by disabling Tone Equalizer, while the gain map could probably be computed from the exposure mask.

2 Likes
41

See also Ultra HDR Image Format v1.0  |  Android Developers

2 Likes
42

Looks interesting! Do you know who is developing it?

I hope this isn’t yet another Google technology that will end up in the Google graveyard.

EDIT: Seems like it is. Some relevant PetaPixel article. The promo picture looks like a good ol’ overprocessed HDR :grimacing:, but this might not be the fault of the technology itself.

43

It’s a Google implementation of the Adobe Gain Map. Basically straight out of the specification, but with a Google GContainer thrown in for good measure. Not sure why they felt the need for that, since it doesn’t seem to do anything that MPF doesn’t already do well enough.

1 Like
44

Triggered by some articles about HDR workflows, I started to take a look at DT capabilities to prepare images for display on HDR displays. Because of the unbound data in the scene-referered workflow I thought, that shouldn’t be a big issue at all and started to read a several threads here.

For testing, I took an image with a large dynamic range and increased exposure so that the brightest non-overexposed pixels are above 255 (using the pipette in RGB mode) without applying tone mapping. Then I played with sigmoids “target white” value and increased it to 400% so that pixels with highest intensity got close to a value 1024. This is what I somehow expected.

Then I exported the image as 10bit avif / jpeg xl using the ‘linear Rec2020 RGB’ profile … well I think I need to read a bit about the different formats and profiles and their compatibility.

The question that I have now, is about the histogram in Darktable 4.6.0. It changes upon changing the histogram profile, but to me it looks like, it is always restricted in values on the x-axis. When I increase exposure, it will not show the entire histogram. But I wonder why, because the work profile is Linear Rec 2020 and values should go from 0 to 1. Is this a limit of the histogram chart or do I make something fundamentally wrong? Is it possible to show the entire histrogram?

I attached a screen shot as well as the raw file with xmp.

Best regards and a happy new year!
Till

P.S. I don’t have a HDR display on my laptop, but consider getting an external one, so I can only judge the results on the phone/tablet.

20230226_165438-_MG_2170.cr3 (25.3 MB)
20230226_165438-_MG_2170.cr3.xmp (30.0 KB)

Screenshot 2023-12-31 154659
Screenshot 2023-12-31 1546591465×850 923 KB

45

This actually causes more problems than it solves, because the output formats are not unbounded unless you’re exporting JXR - which I don’t think darktable can do.

I still can’t understand why darktable users are so afraid of the exposure compensation slider… There’s this unhealthy obsession with “unbounded” when the reality is that any output format IS bounded except for floating-point TIFF which is really only usable for feeding to another piece of software for further processing.

Back when I did use darktable, I did the following:

  • Adjust exposure compensation so that the highlights are just under clipping on the histogram (shouldn’t need to be much, since fundamentally the input device is bounded too. Highlight reconstruction and colorspace conversion could result in some positive gain that causes clipping but generally not muh.)
  • Disable all tonemapping modules (filmic, etc - sigmoid didn’t exist back then) - they’re not necessary for an HDR display
  • Export as linear Rec. 2020 16-bit TIFF
  • Convert to 10-bit Rec. 2020 HLG H.265 video using Experimenting with HLG and HDR10 for still images - #22 by Entropy512
1 Like
46

Actually, any floating point output from dt is unbounded (EXR, JXL, PFM, TIFF).

1 Like
47

I will take a look at the script, to convert DT‘s output to HDR files that are compatible with HDR screens. Thanks for pointing to the discussion!

I am using the exposure slider a lot, but still don’t understand the histogram in DT beyond 8bits/sRGB images. If I increase exposure, the histogram moves to the right and finally it moves out of the shown range. Is the right axis limit set to 1.0 and any value above is not shown?

Well, exporting images as 32bit and plotting the histogram with some Python script will reveal the answer…

Did you change (to which software) because of HDR related issues?

BR
Till

48

Indeed, the histogram currently does not show above 1.0, but not of sRGB, but your display color space. It’s all in the manual.

If you have an idea how to improve this for HDR workflow, feel free to open a feature or pull request on GitHub.

49

Oops I missed that part in the manual. I think first I need a HDR display for my laptop to see how this will change the situation.

50

I don’t know for you, but for me, converting to linear Rec. 2020 16-bit TIFF is not satisfactory, as it will clip the colorspace of my camera, which is a bit larger than ProPhoto RGB. In other words, clipping colors is unacceptable to me, so here is a linear Rec2020 profile with luts instead of a simple matrix. Use it with the perceptual rendering intent. The gamut mapping is done from ProPhoto RGB to Rec2020.

However good that solution may be, it’s highly unlikely that the display at the end of the line will show anything close to the Rec2020 colorspace, more clipping in sight. And from what I’ve seen so far, HDR in the real world is a monumental mess.

My objective for going the HDR route, is mainly a color gamut thing and a little bit for higher dynamic range. But it looks like all this maybe futile.

I’m in search of a solution to bring my (very) wide gamut work to the smartphone world, especially.

Any suggestion?

LUTRec2020-elle-V2-g10.icm (1.3 MB)

51

I am just wondering and your comments on color made me think of the days of my youth when people were buying HIFI sound systems and speakers capable of generating a range of frequencies outside the range of human hearing.

If what we see in the world around us is essentially the reflected light from the surfaces around us and this is or has been quantified at least by one researcher as the so called pointer’s gamut…then why do we need to worry so much about colors that in some cases are so far outside this gamut…

I’m sure there is some nuance to this… I guess it might lie between reflected light and direct light and what we can perceive?? Any way I am sure its one of those dumb questions I should just think on more…

2 Likes