Compressed RAF: help needed.

Hi everyone.

Uwe Müssel has finally PR’d his work on porting libraw’s compressed RAF support into RawSpeed, and it is now avaliable in a raf-compressed branch.

To proceed with integration into the develop branch, among the other things, i need to have a full sample coverage of that code. Thankfully, RPU exists, and it contains all 5 out of 5 compressed raf samples (FIXME: or are there more cameras with that feature?).

However, as the code suggests, such compressed raw file may be 14-bit, and 12-bit.
All these RPU samples are 14-bit. I need one such raw, which is 12-bit.
Specifically, i need this branch to be triggered: https://github.com/Beep6581/RawTherapee/blob/6e55f6bab5ac9289c4bacf2e6cded9b92c3a3d24/rtengine/fujicompressed.cc#L76-L80

I don’t have a complete guide on how to acquire such a sample, i do not know which camera with which settings produces it. I can guess that you can test whether some raw file is the sample i need by adding either derror(); or merror (0, "that is the sample"); into that branch, building RT, looking through the raws, and seeing if some RAF no longer loads.

Perhaps someone can help.

Roman.

1 Like

Maybe @sguyader can help…

I don’t know if I can help. To my knowledge, there are currenty only 5 Fujifilm cameras with lossless raw compression:

  • 4 cameras with X-Trans CMOS III sensor:
    X-100F, X-T20, X-T2 and X-Pro2
  • 1 camera with a regular Bayer CFA:
    GFX 50S

I looked at the official specifications of all these cameras, and on the Fujifilm website it always reports 14-bit raw. I saw no mention of 12-bit raw whatsoever.

There’s one blogger (https://alikgriffin.com/uncompressed-vs-lossless-fujifilm-x-t2/) who said compressed raws are 12-bit, but he failed to give any reliable source about this. He even “saw” differences in image quality between compressed vs. uncompressed… He’s certainly wrong.

This thread mentions 12-bit raw files for the older X-E1 and X-Pro1 cameras, but I can’t find any official Fuji documentation to back that up. And if those raw files are indeed 12-bit, I don’t think they’re a compressed version; they’re just the uncompressed raw files.

X-E1 and X-Pro1 for sure have no compressed RAW format. Only the newest generation of cameras with the new 24 MP X-Trans sensor (the four cameras @sguyader stated) plus the medium format GFX 50s support compressed RAW format. I also never heared about 12 bit compressed RAW, wouldn’t be lossless anyway, so I think the blogger alikgriffin.com is just wrong since there is clearly now difference in image quality between compressed and uncompressed Fujifilm RAWs.

I agree, but the code suggests that 12-bit compressed RAF’s exist…
I would guess it is created by some of the older cameras in high-speed shooting mode (14/12/etc fps)
I did mail the libraw with this question, hopefully they may know whether it’s a dead code not.

I just did some empirical tests with my X-T2: I shot compressed raws, in S (single frame), CL (low-speed burst), and CH (high-speed burst). Bursts are around 8 fps without the battery grip and with mechanical shutter.

Then I compared the file sizes, as I suppose 12-bit vs 14-bit would make the files differ in size:

  • S : 1 file of 23.3 MB
  • CL : 4 files of 23.6, 23.7, 23.4 and 23.7 MB
  • CH : 6 files of 23.7, 23.6, 23.7, 23.8, 22.5 and 22.9 MB

Clearly, there’s no significant difference in file size, so I guess they are all 14-bit.

[Edit] Now a test with battery grip and electronic shutter (burst speed is 11-14 fps):

  • S : 1 file of 23.1 MB
  • CL : 4 files of 23.7 MB
  • CH : 7 files of 23.7, and 1 of 23.6 MB

Same tests, but without lossless compression: all files between 50.5 and 50.6 MB.

1 Like

JFYI:

$ exiftool -bitspersample compressed-raf/*.RAF 
======== compressed-raf/20170114_174341_TFW04727.RAF
Bits Per Sample                 : 14
======== compressed-raf/20170525_0037TEST.RAF
Bits Per Sample                 : 14
======== compressed-raf/DSCF0151.RAF
Bits Per Sample                 : 14
======== compressed-raf/DSCF0527.RAF
Bits Per Sample                 : 14
======== compressed-raf/_DSF5844.RAF
Bits Per Sample                 : 14
    5 image files read

I DO NOT know if that is the field i’m looking for, but if it prints 12, it may be interesting nonetheless.

I get the same result on all my RAF files using this exiftool command.

I’ve got a RAF file from an X10. According to EXIF data, it’s 12 bits per sample. Is this what you need? DSCF1024.RAF (18.9 MB)

The X10 doesn’t compress its raw files. If I’m not mistaken, here we’re trying to deal with compressed raw files support.

Ah, I see. In the exif, it says JPEG under compression, which made me wonder.

Good luck with your search.

1 Like

After a brief mail exchange with the original author[s] of the code, i’m now under the impression that such sample may simply not exist in the wild yet, it is possible that no current camera produces it.

3 Likes
4 Likes

I own Fujifilm X-T3 with bunch of lenses, what can I do? white balance information of X-T3 RAW is not recognized by Darktable at this moment and so I am relying on JPG.

the 2.6rc0 should be of use: darktable 2.6.0rc0 released

@rhonaldjr I have a simple solution: trade your X-T3 for my X-T2, the latter is perfectlly recognized :laughing:

1 Like

My Darktable handles the X-T3 just fine :wink:

Even with profiled denoise and all!

@paperdigits: I am glad, will give a try

@sguyader: nice try, will stick to X-T3 + JPG till the support is released.

@darix: really!!? mine keep throwing “failed to read camera white balance information from…”. I use uncompressed RAW and that could be the reason. Compressed RAW is recognized by DT I believe.