Manual creation of UltraHDR images

Processing
20

I experimented a bit more with libultrahdr today, and I think I started narrowing down the settings required for color accuracy. I used the binary ultrahdr_app directly, since it makes it easier to iterate on the settings.

I successfully ran the app in “mode 2”, i.e. by providing both a “SDR intent” and a “HDR intent”, as well as in “mode 3”, by providing the finished (“compressed”) SDR JPEG as well as the “HDR intent”.

Now, about those SDR/HDR “intents”: ultrahdr_app requires raw RGB images as inputs. I was unfamiliar with the formats, but it seems that those are just simple binary image formats, following a specific layout in memory. They can be easily generated using the ffmpeg program.

For the HDR intent, I first export my HDR edit in Darktable to a JPEG-XL image, using the P3 primaries and the PQ EOTF (I chose P3 since UltraHDR mostly targets recent consumer displays). In practice, I chose a 10-bit depth, quality 99% or 100% (lossless, but some viewers don’t support it so I can’t compare with the end result), and profile “PQ P3 RGB”. Make sure that the width and height of the image are even, by cropping if needed. I then generate the RAW image using:

# HDR intent
ffmpeg -i image.jxl -pix_fmt p010le -f rawvideo image-hdr.raw

(see this issue for the choice of p010le)

For the SDR intent, I proceed similarly, and export my SDR edit to either a finished JPEG or to a lossless PNG/TIFF. Here I use the profile “Display P3 RGB” since we don’t want the PQ EOTF for our SDR intent. If I chose the lossless route, I then have to generate the raw RGB image using:

# SDR intent
ffmpeg -i image.png -pix_fmt rgba -f rawvideo image-sdr.raw

We can now combine the SDR and HDR intents using the ultrahdr_app program. If we already have the finished JPEG, the command is:

ultrahdr_app -m 0 -i image.jpg -p image-hdr.raw \
    -a 0 -c 1 -C 1 -t 2 -M 1 -s 1 -Q 100 -D 1 \
    -w <width> -h <height> -z image-ultrahdr.jpg

If, instead, our SDR intent is a raw RGB image:

ultrahdr_app -m 0 -y image-sdr.raw -p image-hdr.raw \
    -a 0 -b 3 -c 1 -C 1 -t 2 -M 1 -s 1 -q 100 -Q 100 -D 1
    -w <width> -h <height> -z image-ultrahdr.jpg

I used the best quality settings above, but of course you will most likely want to tune them according to your use case.

Let’s briefly dive into the settings:

Settings
  • -i image.jpg: use the finished SDR image image.jpg as the base.
  • -y image-sdr.raw: use the raw SDR image image-sdr.raw as the SDR intent.
  • -p image-hdr.raw: use the raw HDR image image-hdr.raw as the HDR intent. The gain map will then be automatically computed from the HDR and SDR intents.
  • -a 0: the memory layout of the raw HDR image is P010.
  • -b 3: the memory layout of the raw SDR image is RGB8888.
  • -c 1 -C 1: both the SDR and HDR intents use the P3 primaries.
  • -t 2: the EOTF of the HDR intent is the PQ curve.
  • -M 1: the gain map is multi-channel (RGB instead of grayscale).
  • -s 1: the gain map downsampling factor. 1 means that it has the same resolution is the initial image.
  • -q 100: the JPEG quality for the SDR intent (if the JPEG is not used as input with -i).
  • -Q 100: the (JPEG?) quality for the compressed gain map itself.
  • -D 1: use slower but higher-quality encoding.
  • -w: the image width in pixels (must be even).
  • -h: the image height in pixels (must be even).
  • -z: the path to the output UltraHDR file.

These settings give me very decent results. In particular, I get consistent colors, contrast and brightness in the “SDR part” of the image.

Here is a .zip containing a SDR JPEG, a HDR JPEG-XL, and the corresponding UltraHDR JPEG (generated with a higher-quality JPEG-XL than the one uploaded here).
ultrahdr.zip (6.0 MB)

Although this is a significant improvement compared to my previous attempts from two weeks ago, there are still a few issues with this approach:

  • Although I get consistent colors in the shadows and midtones, I observe that the highlights are clearly oversaturated when using a RGB gain map, and I don’t understand why. Conversely, they are undersaturated when using a greyscale gain map (but that makes more sense).
  • ultrahdr_app doesn’t seem to handle noisy images very well when using a RGB gain map: the noise can be greatly enhanced (while the HDR intent looked fine), and the gain map itself looks noisy. This happens even when the SDR/HDR intents are lossless and the quality of the UltraHDR image (and its gain map) are set to 100.
  • I didn’t try to address the EXIF/Google Photos bug.

Let me know if you obtain similar results or if you find a solution to the noisy and oversaturated highlights!

Meanwhile, it seems that things are moving on the Darktable side, with some recent issue proposing to add first-party gain map support.