16 bit float export looks weird in other applications, but 32 bit float is fine

So I’ve began to start using the unclipped export feature of Rawtherapee for when using for raw video processing, and noticed that it 16 bit float gets messed up when imported into other programs such as Davinci Resolve and Affinity Photo, but looks fine when viewed in Rawtherapee.

32 bit float seems to work fine in the other programs, but 32 bit intermediary seems like an excessive waste of space, and would make fitting such intermediaries on my hard drive for big projects more cumbersome. (I already know to transcode to a 12 bit video format, and delete tiffs, even just having them once in high file size is cumbersome).

I know that some of the non - foss program behavior is out of people’s knowledgebase here, but I was wondering if anyone had any ideas.

Example of how it looks. The flatness isn’t the problem as contrast can always be restored. The problem is all the color fireflies I get around the dark areas, as seen in the picture.

Attatched image:

Keep in mind, that 16 bit float has only a 10 bit fraction.

Edit: have a look at the precision limits:
https://en.wikipedia.org/wiki/Half-precision_floating-point_format#Precision_limitations_on_decimal_values_in_[0,_1]

1 Like

If you load a 16-bit integer tiff in range [0;65535] and save it to to 16-bit float tiff in the same [0;65535] range (should be the same for [0;1] range, I just used [0;65535] range for simplification), you will reduce precision for values larger than 2048:

1 Like

Good point. I also noticed the file size isn’t as bad as I thought, especially with lzw compression.

on the other hand, if you use a linear encoding (i.e. “linear gamma”) for your output, with integers you are wasting a lot of bits in the highlights…

1 Like

@agriggio

For that reason I would use 32 bit float as intermediate export format. Even uncompressed to reduce the time to save the intermediate file. SSD is cheap now…

2 Likes

Yes, it’s less precise than 32-bit float (duh).

However, half float is more than adequate for photography - you get approx. 30 stops of dynamic range (good luck representing anything more than 16 stops with uint16) and constant quantization per stop (agreed limited). The quantization error is still smaller than the photon shot noise. As @agriggio points out, there is no need to represent every single integer in highlights because those measurements are noisy anyway (see e.g. why NEF lossy compression works).

Caveat emptor: half float is supposed to be a storage format. If you do many round trips of demanding processing on the intermediate half float data, then I guess there is some small risk the quantization errors might start accumulating… VFX industry (OpenEXR) doesn’t seem to have a problem with it though…

@Waveluke

Are TIFF tags (SMin and SMax in particular) identical beween 16-bit and 32-bit exports?

half-float isn’t a first-class member of the C++ data type collection, so the developer is at the mercy of software implementations. I’ve only dealt with one implementation; are they all IEEE-754?

I used it to provide a 32-bit amenable build of rawproc for lesser Windows machines, but abandoned keeping up with the wonky typecasting and pre-processor #IF#THEN#ELSEs. With that and what @heckflosse has pointed out, I’ll not be using it in my further programming…

True that. As evidenced, more can go wrong with half floats depending on implementation, and is definitely not a mature/established format. But dynamic range & quantization are fundamentally not an issue.

And since the problem appears only in dark areas, you comment also made me think this is what also might be happening. If RT is exporting floats normalized in the [0, 1.0] range (and assuming it does the half float conversion correctly), it might happen that other apps handle half float subnormal numbers (< 2^-14) belonging to deep darks incorrectly on import…

Many apps assume files with 32-bit or 0-1 data to be float. In GIMP, at least you can define what you are importing.

@kmilos helped me understand some of what was going on with Octave: Learning Octave on Windows. I don’t think it was worth the trouble even if I determined a simple workflow for it.

That was related to floating point TIFF interoperability between apps in general, nothing 16-bit half float specific in that particular case.

What I am saying is that apps aren’t able to identify “16-bit float” files or display them properly even if they could. Instead, they go by some unspoken convention and as a consequence aren’t flexible in this area.

Hi @afre, do you have examples?

I don’t think it works that way. Either the app supports and interprets floating point TIFF tags properly (regardless of stored bit width), or it doesn’t. The internal 16-bit<->32-bit conversion step on import/export is (should be) independent of that and should be the only place where things could go wrong. Writing completely separate TIFF code paths that do different things for 16-bit and 32-bit floats would be foolish.

1 Like

Honestly, I haven’t looked into this in a very long time, like a decade or so, except for my questions on Octave. I will defer to your (global your) assessment.

What I have left to say is that people have different ideas of what should be and what happens to be. My wish list is very long but usually things aren’t ideal.

@agriggio PS, though I haven’t used it since CC; get my intel through third party sources now. They seem to waffle on certain features, so things sometimes work and other times break. It depends on how you are importing; e.g. various HDR import methods, file / layer import, import via CR, smart object import, etc.

PS Come to think about it, scripting for PS might be a way around the confusion. One can be more explicit with what one is doing. See https://www.adobe.com/devnet/photoshop/scripting.html.

I have been inclined to believe that 16 bit half float would be fine in terms of precision, and 11 bit for the actual numeral seems to be sufficient for photo and video, given the noise inherent in digital imagery from cameras. I re-opened the 16 bit half float in Rawtherapee and haven’t run into banding issues, even with drastic adjustments.

@Waveluke Would it be possible to share the 16-bit half float image to test in darktable master?

1 Like

Here you go:BMPCC_1_2020-11-22_0713_C0000_000033.tif (12.4 MB)

1 Like