Unknown automatic editing of raw images

Software RawTherapee
1

I recently bought a new camera with the primary objective being to learn how to edit photos and thinking that using raw files would be the most instructive. Furthermore, my expectation is that by working with raw files I’m going to find out what edits (i.e., changes or alterations) are being made to the basic image data captured by the camera’s sensor which will help me to learn how these edits affect the visual appearance of the pictures. However, my initial experience is that the visual appearance varies depending on what software is used to open the raw file. In some cases this variance is quite drastic. To try and illustrate this phenomenon I opened the exact same raw file in several different editors and immediately create a jpeg version that can be uploaded for online viewing. Links to this set of images along with the identity of the respective editor follows:

https://goo.gl/photos/3VSDY8dbMeBVqL3d9 → DPP4 (by Canon)
https://goo.gl/photos/cVd4J2Ysa2Kxvtgq5 → GIMP via UFraw
https://goo.gl/photos/ZDFSHRucmi4y9XoK8 → Picasa (by Google)
https://goo.gl/photos/Uo94ktyz4CheJGqG7 → RawTherapee

The main point here is that I did nothing to alter the file/image. This pretty much implies that the respective editors are making some kind of alteration that differ from each other. The idea that the software developer thinks they know of adjustments that are appropriate for creating a starting point seems quite reasonable and it is understandable that these adjustments may differ from one program to another. With that said it also seems important that the person who is undertaking the editing knows what adjustments (i.e., changes) have been made. If that’s possible I haven’t figured out how to do it with any of the software used for this experiment.

Note: It is my understanding that modern digital cameras are capable of making lots of adjustments and that most users appreciate and value the ability of the camera to do this for them. In that, the camera delivers good pictures with no need for users to do anything. I also like that but it doesn’t help me to know how to edit pictures. With this in mind I should point out that my Canon camera is able to record images in both raw and jpeg format. I’ve done that and there is no difference that I can discern between the appearance that is displayed by the Canon DPP4 editor between the raw and the jpeg file. This suggests to me that DPP4 may be making substantial automatic adjustments to the raw image and I haven’t been able to figure out what’s been changed.

While I can appreciate that this forum is about RawTherapee and knowledge of these other products may have limits I’m assuming that this forum is about the editing of raw files and that I’m not the only one who has had to deal with this phenomenon. My apologies for the verbosity.

2

Have a look here

3

You may not have made any changes, but lots of editors will start by making some base adjustments to try and help you out. I can’t speak to DPP4 (though my understanding is that it isn’t great) nor Picasa (which is not under active development). Rawtherapee applies automatic exposure correction by default, which I generally find to be quite pleasing; you can turn that off by clicking Reset in the GUI. I haven’t used UFraw for a while, but I don’t think it does much by default.

4

@ajax Welcome to the forum!

I haven’t used DPP4 before but I have tried using the original version out of curiosity. To my understanding, DPP allows you to recreate the JPEG that the camera would be able to generate, plus some other convenience features. It is a great way to compare your results from other image editors and processors. You could also save flash card space by only recording RAW files and put them through DPP4 to get the respective JPEGs if you are curious how they would have turned out.

If you observe a RAW file without any changes, you would get a dark grid-like image showing a pattern of red, green and blue pixels; but even then, the camera sometimes bakes in certain things such as noise reduction, and other factors may influence the generation of the raw. But I digress. Normally, if I want to get something closer to a RAW, I would use a command line app called dcraw (website), if you are comfortable with that sort of stuff.

In general, if you would like to get closer to what the RAW would look like and have more control over how the image is processed, I suggest you try researching your camera’s settings; e.g., changing the picture profile to neutral will make the JPEGs flatter and closer to what you would see when opening the RAW file in RawTherapee. At the end of the day, all I have said doesn’t matter as long as you get to the results you like.

5

No apology necessary, you seem to be at a point I was at about 2 years ago. I was very frustrated with the same thing you describe, for lack of understanding what was going on under the hood. I really didn’t start to crack the code until I put aside all the more capable raw processors and did a few exercises with dcraw. You can indeed get the raw measurements out of a camera raw file with it, and then build on the image step by step. I found this G’MIC tutorial quite instructive:

https://github.com/dtschump/gmic-community/wiki/Developing-raw-images-with-G%27MIC

What I came to realize is that all the presets, camera profiles (with the exception of a calibrated camera profile representing the camera’s color gamut to a color-managed workflow) and other “unknown automatic editing” are just contrivances intended to help folks just trying to get a usable image out of their camera. But once you learn the characteristics of the raw data and what the editor tools can do to it, you can throw away all the contrivances and make your images what you want them to be.

With that, I’m compelled to give the standard YMMV disclaimer. For instance, if you’re not comfortable with your operating system’s command line shell, that’s one impediment to my approach.

Welcome to the forum! This is a good place to learn the things that will make digital imaging easier to understand, interacting with knowledgeable folks with a constructive perspective.

6

Exactly. Canon knows best what Canon cameras do to the raw data to come up with the JPEG, so opening a raw file in Canon’s software should perfectly match the embedded JPEG image. Some raw processing programs unrelated to the company Canon try to mimic the out-of-camera JPEG look without knowing the full story of what is done to the picture so the results of these programs range from pretty-close to way off. Some programs, such as RawTherapee, do not try to mimic the out-of-camera look at all. RawTherapee’s niche is that it does as little to the raw data as possible when you use the “Neutral” profile. But here’s the thing. You can’t open a raw file and do nothing to it, because it won’t look like anything. One must always do something to make the data in the raw file resemble a photo - decide on black and white points, demosaic it in some way, use an input profile of some sort (matrix/LUT/curves/other), apply some curve (gamma/sRGB/other), do this in some color space (linear/perceptually uniform/other) and convert between colorspaces in some way. There is no definitive “unprocessed” look because processing is always done to some degree and each step can do it’s task in various ways, and often there is no one clear “right way” of doing any individual step. And these are just the basics. On top of that you can apply exposure adjustments, noise reduction, sharpening, tone-mapping, shadow/highlight recovery and other settings to make the image look better out-of-the-box. Some programs like RawTherapee give you as much control over the process as possible, others like Lightroom do as much for you as possible (e.g. I believe one can’t completely disable denoising and sharpening in LR - it does both to some degree even when both are marked as disabled). In the end each program shows your raw file differently.

4 Likes
7

As already noted, various cameras do various (and different) things to the image data collected by the sensor, before saving the raw file. And then various raw processors “by default” do various other things to the image during and after interpolation to make an “instant pretty” image. (I seem to recall there was a GUI raw processor that tried to start from scene-referred by default, and they got user complaints that the image wasn’t pretty.)

So you can’t get back to “the data that actually entered the camera from the scene” because of what the camera did to the data before saving it to the raw file. But you can produce an image that is “as scene-referred as possible”, meaning you can take steps to eliminate the “make pretty” portion of what various raw processors do by default.

This article has a nice overview of “scene-referred” in the context of interpolating a raw file:

This article goes over the steps to get a scene-referred (as close to as possible) out of various raw processors. It’s an old article (I wrote it), and as boring as watching paint dry, but I think still essentially correct:

https://ninedegreesbelow.com/photography/linux-raw-processor-review.html

If you really want to get into the “nitty-gritty” of what raw processors do (for the interpolation steps, not for the after-interpolation “make pretty” steps), this article outlines what happens when you use dcraw to interpolate a raw file:

https://ninedegreesbelow.com/photography/dcraw-c-code-annotated.html

The above article leads to a second article with the actual code, comments, outlines, and annotations. Nobody but a programmer would want to look at the actual code, but the accompanying outlines and annotations do cover the basic steps of what happens to the raw file during interpolation.

It’s nice that you are interested in starting from “what’s actually in the raw file” as a way to work through various ways to get from raw file to final image. That really does seem to me to be the only way to really learn how to edit images. And yes, at least the older Canon DPP software basically just duplicated what Canon does “in camera” to make the jpeg. My first camera was a Canon and I was terribly disappointed in DPP, though perhaps it’s improved since then. But there didn’t seem to be any way at all to get a truly scene-referred image from DPP.

1 Like
8

Great feedback. Many thanks to all the contributors. It looks like I’ll be busy for a while chasing the many leads supplied herein. However, there can be no doubt that this is a huge plus for RawTherapee.