I wanted to test RawTherapee but all photos from my Fuji X-S10 show up black. All black.
The thumbnails are ok until I open a photo for edit, then they too turn black (I assume it’s because at first the thumbnails embedded in RAF files are extracted and afterwards a better thumbnail is generated from the real data).
I have tried opening a NEF from my older Nikon DSLR and they work fine.
From what I see in release notes support for X-S10 has been added in version 5.9.
RawTherapee version: 5.10, I wanted to test with nightly by it crashes as I open the folder.
One of the files can be downloaded here, along with a screenshot from the app: Proton Drive
Is there some way for me to check logs or get more insight into what’s going on? I assume this doesn’t happen to all users of this camera, otherwise I would’ve found some reports on the Internet.
Does it mean there is no diagnostic log that I can check to learn more about where the app struggles?
Is there some more detailed information about what the app supports? All I was able to find is that this particular camera model is listed among the supported ones. But maybe there are some limitations, like not all settings combinations being supported?
TBH I would expect the OS to be irrelevant at this point - the file data can definitely be read (I can see the embedded thumbnails and EXIF). Once the data is loaded isn’t the processing the same regardless of OS?
Hmm, unless processing works fine and the problem is with displaying the image, like some GFX drivers… but everything works fine for the Nikon files, so we can rule this out. It’s definitely something with processing the Fuji data.
I appreciate the help, it would be amazing if you can provide me with a solution. But I’m happy to put some work into investigating this myself, that’s why I’m asking if there are diagnostic logs and for pointers towards more information.
Got it! The Fuji X-S10 (and probably other Fuji cameras as well) offers three ways of saving raw files: “uncompressed”, “lossless compressed” and “compressed” (I assume with some loss of quality).
The first two work perfectly fine with RawTherapee, the lossy compression gives me the black photos.
Using either “uncompressed” or “lossless compressed” solves the issue. I wonder if this is documented somewhere and I missed it.
I’m a amateur hobbyist so I prefer lossy compression over quality to save on storage. But I can live with lossless compression.
I am happy to stand corrected on this by more knowledgeable people. I don’t understand how any camera manufacturer compresses a RAW file without giving something up. So I always use uncompressed RAW files from any camera. I also feel that storage is relatively cheap now so we shouldn’t worry about the extra cost of storage. There was a time when 1MB of storage on an SD card cost about $1 and that was expensive so we all shot JPGs.
Your time and your photos are too valuable so use the uncompressed version of the RAW files in the future is my opinion. As I say, I am happy for someone who knows more to explain why compressed RAW is better and then I will stand corrected.
Good luck with your problem. I hope it get resolved.
There is lossy and lossless compression. Lossless compression doesn’t throw away any data. You just pay the price of processing power when the camera creates the raw file. I guess you are wondering how anything can be compressed without losing information. Here’s a simple example. At RawTherapee, we ask for white frames which help us figure out when a camera’s sensor clips. This information allows RawTherapee to avoid magenta highlights while preserving as much highlight detail as possible. When we ask for the white frames, we instruct people to compress the images, using zip for example. Many white frames consist of the same pixel values throughout the image. In an uncompressed file, the pixels may be represented by a single number repeated millions of times. Sound inefficient? It’s possible to encode the same data by simply saying that the number is repeated x number of times. The compression that zip or cameras use can do something like that. White frames can compress down to a few dozen kilobytes. Typical images won’t compress as well, but most do compress by a noticeable amount using the same trick. Most cameras have 14 bit raws. There are around 16 thousand possible values per pixel. In an image of many millions of pixels, there are bound to be repeated numbers arranged in some way that can be compressed.
And this statistical property improves drastically if you compress the differences between adjacent pixels instead of the pixel values themselves (with the exception, of course, of the first pixel).
Same concept as a zip file - zip files are losslessly compressed.
As to lossy compression - there are plenty of tradeoffs available for lossy compression of RAWs that have little to no perceptual impact. Lossy compression has gotten a bad rep from the fact that historically, “lossy compression” = “low bit depth JPEG that has already had color transformations applied”.
An example of a basic lossy compression is nonlinear encoding - human visual perception is nonlinear anyway, so you can reduce bit depth with nonlinear (often logarithmic) encoding. Sony does this (and a few other things) with their lossy algorithm, I understand Nikon does too. With logarithmic encoding you could probably drop 14 bits down to 10 without any perceptual loss unless you did some INSANE contrast enhancement
It’s also now possible to do lossy compression without color transformations using higher-bit-depth compression algorithms - see DNG 1.7 which IMO is the “killer app” for JPEG XL as one example.
Thank you, but for me the important thing was to figure out what’s going on. An error popup, even vague like “Unable to decode image”, would’ve been a helpful.
You know: black image → broken app, error message → maybe I can do something about it.
Now that I know how things are I can switch to lossy compression and give RawTherapee a try
There’s always a trade-off, but it isn’t always symmetric. Sometimes you gain much more than you lose. The improvements in saved space and transfer are well known. Others have explained about compression and its cost in details. There is one more practical consequence.
It takes some processing power to compress all this data, so you might think: it’ll make saving the photos slower and will use more battery because there’s more work to be done. In practice I would expect the compression to be very fast and efficient (the chip and algorithm should be designed for each other). But more importantly there is another component involved: the SD card. It is the slowest so compressing the data actually makes saving each photo take less time.
When shooting burst this translates to longer bursts because the memory buffer can take more compressed photos before filling up and slowing down. Then the SD card can write each photo from the buffer in less time. So overall more photos can be taken per amount of time. For some photography types this may be important.
With lossless compression I think the cost is only in some camera complexity and is paid for by the manufacturer (it should be one time per camera generation so I doubt it even affects the camera price).
And in return you get better burst mode, save storage, transfer time and so on.
With lossy compression you lose some quality (but would you really notice?) and the gains are larger.
Of course in the end the choice is dictated by your needs.
I’m a hobbyist, none of the above has any practical meaning to me. I take vacation snaps and then scale them down for the internet anyway
But I keep all my RAWs on an HDD for ever (with backup). I can’t be bothered to delete the bad ones, so I prefer that they take less space if possible.
That is not true. It still costs you time and money to get places, that is an investment all by itself.
My photography teach instilled upon us that film and chemistry are the cheap parts of photography. Your time, travel expense, etc are far more valuable and scarce.
