Inspired by this thread (Processing RAWs for HDR displays in Darktable), and by some UltraHDR pictures a friend took, I spent some time recently digging into existing tooling for generating these types of pictures, and wanted to share my progress in the hopes that people with more understanding can improve my process. I did a lot of searching around to see if there were any other tools available for manual creation of UltraHDR photos, and couldn’t find much, so hopefully this is helpful to some people!
Background
UltraHDR: a high dynamic range file format proposed by Google. My understanding is that this is similar to the proposed Adobe Gain Map spec, but I haven’t confirmed this. Put simply, this format stores two JPG images (an SDR rendition and a gain map) in a single file using the existing CIPA Multi-Picture Format spec. When displayed on an SDR display, or with a program that doesn’t support UltraHDR, the SDR rendition is used. The main source I’ve seen for this type of image is from Google Pixel phones, which has a toggle in the camera to create them.
Programs: I’ve been displaying these images on (a) an iPhone and a Pixel using the Google Photos app, and (b) an M-series MacBook Air in the Chrome browser. Your mileage may vary for display support. Greg Benz Photography has some good resources on support, and a testing page.
Tooling: Once you have an UltraHDR image file you may want to check parameters or other things. The embedded XML data is viewable in a plain text editor, you just need to scroll until you find it. You can also extract the gain map image using exiftool with the following: exiftool -b -MPImage2 ~/UltraHDRImage.jpg > GainMap.jpg.
Examples
(For the “screen” column, taking a screenshot on MacOS preserves the displayed brightness ratio in a SDR jpg, which is helpful for demonstration, although it doesn’t look exactly the same as it does in person.)
Attribution: First example is own work and licensed CC0. Bridge photo by chaimav from Take the High Road and licensed CC BY-NC-SA 4.0
Update: When viewing my post, the embedded HDR images don’t render correctly in-browser, probably due to metadata changes after uploading. Here are the original HDR image files for those that are interested.
hdr_examples.zip (1.4 MB)
Merge Process
To generate these photos, I used libultrahdr-wasm, a WebAssembly build of libultrahdr. I was able to find a built version of the library on NPM in @monogrid/gainmap-js. I then wrote a small script which allows me to manually specify the HDR parameters. The only dependency here is Node. The libraries are licensed under Apache 2.0, my example code is too simple for a license, but 0BSD if it matters. This is also a “Works on My Machine” situation, best of luck!
Here’s the step-by-step:
- Download
libultrahdr-esm.wasmandlibultrahdr-esm.jsfrom@monogrid/gainmap-js/libultrahdr-wasm/build/in the linked NPM repository. - Rename
libultrahdr-esm.jstolibultrahdr-esm.mjs. - Create a new file
hdr.mjswith the contents of the script below. - Edit
imgName,gainName,outName,width, andheightfor your files. - Run using:
node hdr.mjs
hdr.mjs Script
import { readFile, writeFile } from 'fs/promises';
import libultrahdr from "./libultrahdr-esm.mjs"
const libraryInstance = await libultrahdr()
const imgName = './DSCF4682_sdr.jpg'
const gainName = './DSCF4682_gain.jpg'
const outName = './DSCF4682_hdr.jpg'
const width = 1749
const height = 1080
const img = new Uint8Array(await readFile(imgName));
const gain = new Uint8Array(await readFile(gainName));
const metadata = {
"gainMapMax": 1.4888443464573364,
"gainMapMin": 0,
"gamma": 1,
"hdrCapacityMax": 1.4888443464573364,
"hdrCapacityMin": 0,
"offsetHdr": 0.015625,
"offsetSdr": 0.015625,
}
const result = libraryInstance.appendGainMap(
width, height,
img, img.length,
gain, gain.length,
metadata.gainMapMax, metadata.gainMapMin,
metadata.gamma, metadata.offsetSdr, metadata.offsetHdr,
metadata.hdrCapacityMin, metadata.hdrCapacityMax
)
await writeFile(outName, Buffer.from(result))
Gain Map Generation
For photos I’ve been taking a very manual approach. I create the SDR rendition in Darktable as I normally would. Then I create a duplicate, and tweak the contrast and exposure until I’m happy. I sometimes use tone equalizer, and usually make it black and white (although the spec allows for colored gain maps, which create an interesting effect).
I’m sure I could create a preset or similar to make this faster, I also imagine there’s a more technically correct way of generating these. I’d be very interested in suggestions or ideas on how to create a more automated workflow. There’s a lot in the UltraHDR spec on how to generate values that I haven’t gone through in detail.
Conclusion
For future work, it would be cool to build a little web tool to merge an SDR and gain map together, with sliders for the different values. The processing is fairly quick, and all local, so I imagine it could update the preview fairly quickly. This isn’t something I’m as familiar with, but it certainly seems possible. Also, a darktable preset for generating gain maps combined with a better CLI wrapper could allow for a nice workflow with darktable-cli. It’s also possible store the gain map in a lower resolution than the main image, but I haven’t tried this yet.
So far, it seems a bit like a novelty to me, and I probably won’t start exporting all of my photographs in UltraHDR. Visually, the main indicator is that it makes the highlights of the photo brighter than the system UI white. When the effect is high, it can be quite striking, a real “turn it up to 11” type of look.
Some high contrast photos (in particular of lit up buildings at night) are very cool, but for most of the pictures I take, it’s a pretty minor change. I really only notice the difference when my SDR images are in a shared album next to somebody else’s Pixel phone photos. When you swipe from an HDR to an SDR photo, it makes the latter look very dull in comparison.
