Output Rounding


#1

I was curious about how G’MIC rounds at clipping boundaries; e.g., when saving the floating-point buffer to an integer PNG. In the stress test below, the command yields a PNG within the range of 1-255. If we remove one 9 digit from each value, the range shifts to 0-254.

gmic (.99999999,254.999999) -o tst.png tst.png

I guess, in practical terms, this doesn’t mean all that much. It is just that over time, there would be a clipping creep in the shadows in saves or in environments that aren’t by default floating-point.


#2

I believe that’s due to cast from float to int, which is in effect a truncate. Filters do have the option of using “-round 1” but that’s left entirely to the author.


(David Tschumperlé) #3

@garagecoder is right. If the output image format manages only (unsigned) integer values, then a simple C-style cast is performed when saving the image data.

Command -round can be used to avoid that, and is not done by default because it takes times, and doing it by default would also try to round values that maybe do not need to be rounded after all (if all your image pixels are already integer values for instance). A cast is the minimal necessary operation that has to be done.


(David Tschumperlé) #4

That also explains why in some subtle cases, you may get strange pixels saved, when your values are outside the supported range [0,255] (or [0,65535] in 16bits PNG).
For instance, a value of -1 will be cast as 255 (so a low pixel value becomes a bright pixel!), or why 256 will be converted as 0 (the opposite effect), when saving an image into a 8 bits file format.
When saving a file, it’s not a bad idea to add commands -round -cut 0,255 to make sure your initial float-valued pixels will fit well with the file format constraints.