Manual creation of UltraHDR images

[Krzysztof_Kotowicz]

That’s cool @JLTastet !

In the meantime I added support for supporting other ultrahdr_app encoding modes for the Lua plugin, I’ll see if I can code in mode 2 and/or 3 according to your instructions and we’ll go from there It should be simple, though the “dimensions need to be even”, and that you need to specify dimensions on the ultrahdr_app command line sound strange (I guess their intent format is just a stream of pixel values with no metadata?).

EDIT: Requiring image even dimensions was just fixed in enable support for odd dimensions for rgb inputs by ram-mohan · Pull Request #277 · google/libultrahdr · GitHub.

On a more practical note though - how do you edit the HDR file in DT, later exported to JPEG-XL? I suppose it’s not just different export settings for the SDR and HDR, but different modules, color profile settings etc. Generating a monochrome gainmap is dead simple, whereas the full HDR editing in DT is a mystery to me.

The reason I’m asking is to figure out whether some of this could be automated, or stored in a preset.

[Donatzsky (Nis Donatzsky)]

@Krzysztof_Kotowicz

On a more practical note though - how do you edit the HDR file in DT, later exported to JPEG-XL?

Don’t have an HDR monitor, so no personal experience, but these two threads should help you with that:

Processing RAWs for HDR displays in Darktable

Lately, I have been experimenting with “display HDR” photography¹, i.e. producing files which, when viewed with a compatible viewer and a compatible display, produce an image with a potentially much higher contrast (and larger gamut volume) than what is achievable with a SDR display or a SDR format like JPEG. ¹Yes, we need a better (and more easily searchable) name for that, that distinguishes itself from what is usually referred to as “HDR photography” i.e. HDR tone mapping. I await your sugge…

Exporting images for use in HDR TV production

Hi, I am trying to find out if darktable can output images in an HDR format for use in HDR TV productions that are graded in Da Vinci Resolve. I would think that using the HDR output color profiles would accomplish this, but so far I haven’t had too much luck. I have tried to export images in linear Rec.2020, HLG Rec.2020 and PQ Rec.2020, as exr and 16 bit tif. As far as I can think, I would then be able to take these images into Resolve, assign their respective input colorspace/gammas and the…

[Krzysztof_Kotowicz]

@JLTastet

Here is a .zip containing a SDR JPEG, a HDR JPEG-XL, and the corresponding UltraHDR JPEG (generated with a higher-quality JPEG-XL than the one uploaded here).

@JLTastet, do you have the raw file and the XMP sidecar file as well, maybe? I’m trying to find a workflow that would be the most convenient in DT with the plugin, and a complete image set would help me.

I made a total overhaul of the plugin (not yet comitted upstream), moving it out of the export UI to a separate UI component, and fixing several issues:

Alas, I’m facing a major block now. DT Lua APIs don’t allow me to modify export’s “global options”, including the color profile. In fact, it’s currently impossible for Lua plugins to export two images (SDR and HDR, in our case) with different color profiles, which your workflow depends on.

I can workaround that by supporting a two step approach, in which exporting some or all of the source files is left to the user, and the exports are then reimported into DT for the plugin only to merge them into UltraHDR (and correct the EXIFs?). Or I could try to figure out how to generate the exports outside of DT. Any preferences?

[Krzysztof_Kotowicz]

Good news! Pull Request has the newest plugin code, with support for generating UltraHDRs from SDR and HDR pairs.

The HDR needs to be a JPEG-XL file with the appropriate color profile, likely coming from a manual DT export first. The SDR can be of any DT-supported format. The UltraHDR step uses -q 95 -Q 95, as the example file is then 3x smaller (quality setting could also be exposed in the plugin UI).

The example output UltraHDR attached below.
desert_ultrahdr_04.jpg.zip (4.9 MB).

[wpferguson (Bill Ferguson)]

There is a way to modify the global parameters using darktable.gui.action()

[Krzysztof_Kotowicz]

Alas, this doesn’t seem to work for the color profiles combobox in export I tried all possible variants I could think of to set Display P3 RGB there programatically:

dt.gui.action("lib/export/profile", 0, "selection", "item:Display P3 RGB", 1.000)
dt.gui.action("lib/export/profile", 0, "selection", "Display P3 RGB", 1.000)
dt.gui.action("lib/export/profile", 0, "selection", "item:26", 1.000)

     7.7990 [dt_action_process] effect 'item:Display P3 RGB' not valid for action 'lib/export/profile'
     7.7991 [dt_action_process] effect 'Display P3 RGB' not valid for action 'lib/export/profile'
     7.7992 [dt_action_process] effect 'item:26' not valid for action 'lib/export/profile'

[wpferguson (Bill Ferguson)]

dt.gui.action("lib/export/profile", 0, "selection", "next", 10.0)

assuming you are on the first item in the list

[Krzysztof_Kotowicz]

@wpferguson

dt.gui.action("lib/export/profile", 0, "selection", "next", 10.0)

That works, but has its significant downsides. I filed #522 to track this, not to derail the thread.

[JLTastet]

@Krzysztof_Kotowicz

@JLTastet, do you have the raw file and the XMP sidecar file as well, maybe? I’m trying to find a workflow that would be the most convenient in DT with the plugin, and a complete image set would help me.

Sorry for the slow reply, I was quite busy last week, and out taking pictures last weekend

I have uploaded to my Google Drive the raw and XMP files for 6 example images, as well as the JPEG/JXL/UltraHDR files derived from them: ultrahdr_test_with_raws.zip - Google Drive (228 MB)

Each raw has two XMPs: one for the HDR version and one for the SDR version. I have also produced two UltraHDR files for each image: one with a greyscale gain-map and one with a RGB gain-map.

Regarding the latter, I still haven’t resolved the issues of highlight over-saturation and bright noise artefacts when using a RGB gain map, but I did make some progress.

• Over-saturation (see the “sunset” example): I have noticed that the more desaturated the highlights are in the SDR image, the more saturated they will be in the UltraHDR file. So it seems that the gain map is currently overcorrecting. Could be an issue with my parameters, or an upstream bug in libultrahdr.
• “Bright noise” (see the “bridge” example, both with and without denoising): In high-ISO or strongly pushed images (e.g. exposed for highlights), the noise is sometimes greatly enhanced in the UltraHDR image, producing many bright, saturated dots. This does not occur in the JPEG XL, so it is not an effect of HDR per-se. This phenomenon seems particularly frequent around blue specular sources reflecting off a nearby surface (and to a lower extent around other bright primaries), hinting at a gamut issue.

I will see if I can narrow-down what causes these issues, and if they are related.

@Krzysztof_Kotowicz I will also try to test your PR when I have some time. But the image you sent matches mine (apart from stronger JPEG compression), so at least for greyscale gain maps it seems to do the right thing. I would support exposing at least the downsampling parameter to the user, and maybe the gain map quality as well (defaulting to the same as the base JPEG). But I wouldn’t include RGB gain maps for now, or mark them explicitly as “experimental”, until the above-mentioned issues are understood and resolved.

[Krzysztof_Kotowicz]

Thanks, these are great!

I had to tweak the XMP files a bit for DT to recognize all of them on import (adding darktable:version_name="sdr" flags to label them, and renaming to original_file.ext.xmp and original_file_01.ext.xmp), but the current plugin (code in PR) can successfully process them in SDR + HDR mode! I didn’t look deeply into the resulting files, but at least they are valid UltraHDRs, generated straight from RAWs.

The only tricky part is to make sure that the first selected image is the SDR (or add hdr tag to the hdr one).

Regarding encoding:

• HDR intent is exported to PQ P3 RGB JPEG-XL (quality=100, bpp=10), followed by -pix_fmt p010le,
• SDR intent is exported to Display P3 RGB 8bpp PNG, followed by -pix_fmt rgba.
• The files are cropped to have even dimensions by ffmpeg.

I’m thinking of whether to ditch JPEG-XL, as it seems quite slow to generate. In theory, any lossless format that ffmpeg could process would do, no? Is there any advantage for JPEG-XL as an intermediary format? Are we also OK to hardcode both P3 and PQ EOTF?

Regarding visuals:

I’ll defer to you for the actual artifacts in the outputs and the best flags for the binaries; my monitor here barely qualifies as an HDR one, so I can’t easily spot all the visual problems with the outputs. I’m currently using multi-channel gainmap (-M 1) in SDR+HDR method, though the output image actually has gainmap metadata that seems to be flattened to one channel (that might be an upstream bug?):

$ exiftool ../koto_output/bridge_ultrahdr.jpg -MPImage2 -b | exiftool -XMP -b -
<x:xmpmeta
  xmlns:x="adobe:ns:meta/"
  x:xmptk="Adobe XMP Core 5.1.2">
  <rdf:RDF
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
    <rdf:Description
      xmlns:hdrgm="http://ns.adobe.com/hdr-gain-map/1.0/"
      hdrgm:Version="1.0"
      hdrgm:GainMapMin="-13"
      hdrgm:GainMapMax="9.52432"
      hdrgm:Gamma="1"
      hdrgm:OffsetSDR="0"
      hdrgm:OffsetHDR="0"
      hdrgm:HDRCapacityMin="0"
      hdrgm:HDRCapacityMax="5.62238"
      hdrgm:BaseRenditionIsHDR="False"/>
  </rdf:RDF>
</x:xmpmeta>

Exposing downsampling and/or gainmap channels to UI would be trivial; quality is already exposed (though I’m using the same setting for both gainmap and the main image).

BTW, the HDR renditions in DT look quite dark to me, as you dialed down their exposure. This causes the output gainmap to have relatively strange parameter (hdrgm:GainMapMin="-13"), Intuitively, HDR intents should appear as overexposed, as they define the “boost” over the SDRs, no?

[agriggio (Alberto)]

Hi,
I’m also playing with libultrahdr, but unfortunately I do not have a system capable of displaying the HDR version of the image.
So, I wondered if there’s anyone who could take a look at the attached pics (zipped to avoid possible manglings of metadata) and tell me if they look reasonable?

Thanks in advance in any case!
test-uhdr-images.zip (3.5 MB)

[kmilos]

@Krzysztof_Kotowicz

I’m thinking of whether to ditch JPEG-XL, as it seems quite slow to generate.

Tried dropping the “effort” parameter?

[JLTastet]

@Krzysztof_Kotowicz

Is there any advantage for JPEG-XL as an intermediary format?

I think mostly file size and colour profile metadata (unlike e.g. EXR). But since this is just a temporary file, 16-bit TIFF (or even 32-bit floating point) might be just as good. Even EXR might be okay, since we are managing the colour profiles by hand anyway. The main reason I used JPEG XL above is that I also wanted to store the HDR intent.

In case we want to keep JPEG XL, I think it might be sufficient to export to a visually lossless file, instead of a mathematically lossless one.

@Krzysztof_Kotowicz

I’m currently using multi-channel gainmap (-M 1) in SDR+HDR method, though the output image actually has gainmap metadata that seems to be flattened to one channel (that might be an upstream bug?)

Interesting… I will have a closer look. For now, I would rather use a single-channel gainmap (-M 0). I think this is also what Lightroom does.

@Krzysztof_Kotowicz

BTW, the HDR renditions in DT look quite dark to me, as you dialed down their exposure. This causes the output gainmap to have relatively strange parameter (hdrgm:GainMapMin="-13"), Intuitively, HDR intents should appear as overexposed, as they define the “boost” over the SDRs, no?

Your intuition is right, but currently Darktable seems to map [0, 1] RGB values to [0, 203] nits when using a SDR transfer curve (e.g. Display P3 RGB), but to [0, 10000] nits when using the PQ transfer curve. So the image is interpreted differently depending on the export parameters, and if I want the shadows/midtones to match between the SDR and HDR intents, I need to rescale the HDR intent by applying an exposure compensation of log2(203 nits/10000 nits) = -5.62 EV.

This is clearly suboptimal, and ideally Darktable would reason in terms of nits and display values > 203 as HDR. But this is an issue with HDR support, and not specifically with gain maps. So until this gets sorted upstream, I think we need to continue to apply the scaling.

@agriggio

So, I wondered if there’s anyone who could take a look at the attached pics (zipped to avoid possible manglings of metadata) and tell me if they look reasonable?

They do! Although the effect is very subtle in the “dinosaur” and “sunset” images, and completely invisible in the “statue” one (I think because it entirely fits within the SDR range).

[kmilos]

@JLTastet

This is clearly suboptimal, and ideally Darktable would reason in terms of nits and display values > 203 as HDR. But this is an issue with HDR support, and not specifically with gain maps. So until this gets sorted upstream, I think we need to continue to apply the scaling.

That’s right. It’s important to understand that dt is built around ICC profiles, and in those 1.0 is simply max (white) value, there is no concept of “1.0 is diffuse white, and above is HDR”. It’ll be interesting to see how ICC evolves as well…

[agriggio (Alberto)]

Hi,

@JLTastet

They do!

Thanks a lot for testing!

Although the effect is very subtle in the “dinosaur” and “sunset” images

This matches my expectations more or less, but:

and completely invisible in the “statue” one (I think because it entirely fits within the SDR range).

This doesn’t. I was expecting a big chunk of the sky to get brighter…

So, let’s try again: how about these?

test-uhdr-images-2.zip (4.0 MB)

Thanks in advance!

[JLTastet]

These are indeed quite a bit brighter (even for the SDR rendition), and I can now see the effect of the gain map for all three of them.

[agriggio (Alberto)]

Great, thanks!

@JLTastet

even for the SDR rendition

Well the idea is that the two should give a similar “feel” (i.e. the SDR should be a reasonable approximation of the hdr one when all you have is a non-hdr display pipeline)

[Krzysztof_Kotowicz]

@kmilos

Tried dropping the “effort” parameter?

That helped, thanks! FWIW, I tried TIFF and PNG as intermediaries, and there’s something special about JPEG-XL. Whereas ffmpeg -pix_fmt p010le -f rawvideo produces the same outputs for lossless TIFF and PNGs with “PQ P3 RGB”, its output for JPEG-XML is different, binary wise. Since it works for @JLTastet visually, I’m keeping it as an intermediary format then.

[g-man]

What number are you using? I think for lossless you can easy go to 2 or 3.

[Krzysztof_Kotowicz]

After the change, effort = 1.