Confusion about cropping of masked pixels in raw images

Processing
,
1

I am trying to understand the raw image processing process. I noticed that images from my Sony A6700 are differently sized out-of-camera compared to what Darktable produces.

To be precise, the raw dimensions of the images are 6656x4608. This includes a black border on the right and bottom, as well as a column of ‘duplicated pixels’ on the right, on the inside of the black border:

_DSC1863
_DSC18636656×4608 4.5 MB

The metadata in the raw image specifies the following crop area:

DefaultCropOrigin 26 20
DefaultCropSize 6192 4128

If I understand correctly, this is how the raw image should be cropped, according to Sony. This then results in a 6192x4128 image, that is cropped from a point (26, 20) from the top left to from the bottom right (438, 460).

Now, RawSpeed’s README says:

RawSpeed does NOT crop the image to the same sizes as manufactures, but supplies biggest possible images.

OK, fair enough. In practice, Darktable crops the A6700 images to 6244x4168, by cropping (412, 440) pixels from the bottom right (and nothing from the top left).

Now, what I am confused about is how RawSpeed determines this. In RawSpeed’s cameras.xml, there is this line for ILCE-6700:

<Crop x="0" y="0" width="-28" height="0"/>

This would suggest that it would only crop 28 pixels from the right. This would correspond with the strange column of ‘duplicated’ pixels, I suppose. Does that mean that Darktable crops off the black border automatically?

Moreover, is it even valid to deviate from the official crop dimensions and cropping to the biggest possible image instead, considering for example lens correction which is now applied to a bigger image than it ‘should’?

Lastly, purely out of curiosity, why do raw images even include this “masked pixels”? Is there any benefit in including them in raw images? Or is it just an artifact of how image sensors work?

2

Howdy, welcome to the forums.

A developer gets a sample file and inspects it.

All darktable users say “yes” :wink:

3

I get that is where the data in cameras.xml comes from, but I meant more in the sense of how

<Crop x="0" y="0" width="-28" height="0"/>

in cameras.xml translates to cropping (412, 440) pixels from the bottom right. RawSpeed does not use the image’s metadata at all, correct?

I assume Darktable/RawSpeed automatically crops off the black border, and the 28 horizontal pixels take care of the column of ‘duplicated’ pixels?

4

My Canon G16 sometimes has ‘strange’ pixels on the border, but not always. I am happy to throw in the extra cropping step to maximize the size of the picture captured. I know with my Olympus TG6 that the out of camera JPG crops a huge amount of the pixels and that can often be detrimental to the image. So I see this issue is more about personal taste than having to follow the manufacturers lead.

You may be able to create a preset in the cropping module that crops out the masked pixels. Possibly a % setting would work. This crop could also be auto applied as a style to your images if you want to achieve something like the manufacturer does. The beauty of DT is that this is your choice to do rather than having it forced upon you.

image

5

The masked pixels are correctly cropped out by the raw black/white point module. I’m just trying to understand the process.

Note that this is different than using the crop module, because the cropping in the raw black/white point module takes place early in the processing pipeline, e.g. before lens correction.

6

My confusion, I thought it was similar to my strange edges on the G16 which need cropping out in some images but not all.

7

The crop property in RawSpeed for Sony files is for uncompressed and lossy compressed only (as those were the only two types when the decoder was written way back when). If you check, you’ll see those don’t have the superfluous black bars, just the duplicated right border pixels.

The black bars in the “new” Sony lossless format are not real and do not contain anything useful. They’re just manual zero padding to make the raw image an integer multiple of 512 (as this new mode also switched to 512×512 tiles in the TIFF-like container). This is a faux pas made by someone at Sony when coming up with this new lossless mode, as this manual padding is completely unnecessary, the TIFF spec says both readers and writers should deal w/ this padding automatically. So Sony had to add more metadata instead to track what the valid raw image area actually is (not to be confused w/ their preferred crop), and RawSpeed does use this metadata in this particular case.

For the interested, all of this can be found in the source code and GitHub commits and issue discussions.

8

Thanks, that clears up a lot!

So Sony had to add more metadata instead to track what the valid raw image area actually is (not to be confused w/ their preferred crop), and RawSpeed does use this metadata in this particular case.

I guess this metadata is not in the EXIF, but somewhere else? (I can’t find anything else in the EXIF related to cropping except the DefaultCropOrigin and DefaultCropSize.

For the interested, all of this can be found in the source code and GitHub commits and issue discussions.

Of course, but it can be rather difficult to find, especially if you’re not familiar with the code base and you don’t know exactly what you’re looking for :wink:

9

It is in some new/special Sony tags. W/ a recent one, try exiftool -a -u -s G1.