I’ve just acquired a new camera: Fujifilm X-T30. I use Lightroom (6.14) for routine processing of raw files and try to reproduce similar results in dt 3.2.1 as a learning exercise. My version of Lr doesn’t understand raf files from the X-T30 so I have to convert to dng. When I first loaded the dng images I took yesterday in Lr, exposure was about right, but when I loaded the same images into dt, as either dng or as raf, I had to increase the exposure by between +2.8 and +3.1 EV (even after applying lens correction) to have the images appear with a comparable level of brightness. That is an enormous amount - the camera isn’t that wrong, according to other tests I have made.
This effect does not occur at all with the previous X-T20 camera. I assume there is some preset that needs to automatically applied to get the brightness (correct term ?) equivalent between Lr and Dt for X-T30 files. Is this correct?
When you opened those images in the darkroom view, were either basecurve or filmic applied? if neither of those is active, a raw image will indeed look too dark (and flat).
I’ve seen some “funny” differences, mostly depending on whether or not there was a sidecar file available for the image at the moment of importing it in dt… (that is for dt 3.2.1).
Ah, thanks for this. The base curve was not applied and even then I had reset it to a straight line (because, earlier (weeks ago), it didn’t seem to give the acceptable results). Having now adjusted the base curve to ‘fujifilm-like’ things are much ‘improved’. Applying filmic-rgb makes a very small additional lightening of the image. Now to get the brightness, at the numerically same temperature, to appear to my eyeball to be the same as Lr, I have to uplift the exposure by only 0.5 EV, compared to the previous 2.8 to 3.1.
While this is a great improvement, I still don’t understand why these two raw processors differ by half a stop , which is still a discernible amount.
No sidecars files are present (aside from those created by dt when importing the folder).
Usually, you’d use either a basecurve or filmic, not both, as they are based on completely different approaches.
There is an article on this site that might help you understand the differences: https://pixls.us/articles/darktable-3-rgb-or-lab-which-modules-help/. It’s written by the author of the filmic module.
As for the visible difference: the basecurve tries to simulate the out-of-camera jpeg, soe of those are rather aggressive adding contrast and brightness. Filmic is much more gentle in that respect, and often needs 0.5-1EV extra exposure. But not using LR, I can’t say anything about the difference wrt that. Then again, no two raw developers give the same result, anyway… (whether discussing programs or users )
I’ve been meaning to capture an illustration of such for a while; now seems like a good time to do so…
I dug out the ole ColorChecker and made a shot of it metered on the fourth gray patch from the left, which a lot of folk say is close to middle gray. Here’s a screenshot of that raw developed only to “linear RGB”, that is, in the original light energy relationship of the raw data:
Actually not a bad render, except that it’s a bit dull (I hope the screenshot captures that). Note that the whole scene is evenly lit; really, not many of our scenes are like that without some supplemental lighting. Now, let’s look at it with a tone curve applied:
I went a bit contrasty with it, pulling down the lowest part and steeply raising the middle; note the curve plot in the lower-left pane.
Thing about that tone curve is, it’s quite a bit subjective. That the in-camera JPEG looks good is because some engineer (or worse, a programmer ) at the camera manufacturer picked a curve they thought looked okay, in line with the name of the Picture Control (or whatever your camera calls it) in which it resides. Same thing with the dt default tone curve for your camera, moreso in that it’s probably based on the camera’s curve. In all the raws I’ve processed, only one or two looked nice “right out of the box”, or in the original linear. A recent PlayRaw comes to mind… All the others required some curving. Right now, I’m experimenting with mixed-light scenes and how much lift is do-able, for instance, to bring an interior close to an exterior window, and those curves can be quite aggressive:
An in-camera JPEG has applied a particular tone curve, but it’s only one of thousands that will look decent. Once one comes to appreciate that fact, and then to control it’s ways, a lot more control over their renders can be had.
If the camera applies some kind of dynamic range (DR) magic, it’ll underexpose (to save the highlights), and apply some processing to brighten the image (midtones and shadows). Lightroom may do the same (it may be able to interpret some metadata); darktable cannot do that.
But don’t worry: raise exposure (even if you see clipping), disable the base curve, and use filmic to unclip your highlights. (Turning off the base curve and enabling filmic + automatic exposure compensation will be done automatically if you set darktable to use the scene-referred workflow in preferences. By default it uses the legacy display-referred processing because that’s what most other software does and what new users are familiar with.)
You may even save a new exposure preset to be applied only for the X-T30.
Start with
Then read
and watch videos on YouTube (there are many more, but some, like Robert Hutton’s series that are also mentioned on the darktable ‘resources’ page, are outdated):
If you post problematic images, the community will probably be able to help. And you can check the PlayRAW topic: many people demonstrate developing images in different ways, using different software.
This is an old thread by now, but I wanted to highlight this. A lot of people seem to not realize this.
Many cameras these days ‘sneakily behind your back’ underexpose a bit, and write a bit of metadata. Programs such as Lightroom or the manufacturers own RAW-processor can read this and know they need to boost it a bit. In doing this, they capture extra information in the highlights because the shot is technically underexposed… and the sensor is clean enough that shadows can be easily boosted.
Fujis do this - from what I read on DPReview - if one of the DR modes is engaged.
Another example is one of the Fuji GFX cameras where above a certain ISO setting, the camera just doesn’t apply any extra gain at all. Iso 3200, 6400, 128000… they are technically the same shot, nothing changes in the sensor settings. Just a bit of metadata is written to the file ‘this should be made … stops brighter’.
It can do wonders if you look at ‘wow, how much highlight information is captured even at ISO 128000!’. It can be a PITA if you think at ISO 3200 ‘ow wow, I can boost the shadows by +4 EV and everything is still clean!’ and then at ISO 128000 you notice that it can only be boosted by two stops… (because the camera ‘sneakily’ underexposed by -2 EV).
It just shows to either 1) ‘know your gear’ or 2) don’t worry about these things at all.
The OP for example noticed ‘oh the default is dark, what is going on’. I would be more inclined to think like ‘Hey, I need to boost it a lot. Oh, it still looks good… ’.
I’ve said this before here and there, but I never understood why people want to start in a RAW editor with the exact same image the camera displayed on a screen when the picture was taken. I basically only use it for checking focus.
And if I read posts about people expecting the RAW file to look just like the in-camera JPEG, I still think those people don’t really understand what a camera RAW file actually is.
Well, I’ve never seen a camera manual explain how they treated the sensor data to get a particular ISO setting. So is it more dishonest than not disclosing the loss of dynamic range in cases where the signal is amplified?
You’ll still get the shadow noise, but they’ll have a bit more room to play with in the highlights (theoretically at least)
I think that has to do with many cameras being ‘ISO invariant’. I believe the term basically means (is supposed to mean?) that increasing the amplification of the signal that enters the analog-to-digital converter (raising ISO) has no benefit over applying the same amplification via multiplying the output of the converter in post-processing. There are many discussions about this with arguments pro and contra. If this is a true feature, then camera manufacturers are not being dishonest; in fact, they make it possible to avoid highlight clipping.
If you think about it, if you have two cameras that use sensors of very similar characteristics, but different base curves, their JPG output of the same scene could be dramatically different. And the ISO standard says that the ‘ISO sensitivity’ is to be determined from the sRGB output of the camera, not from the raw file. (Film speed - Wikipedia)
Like I said in my post, this gives more headroom in the highlights, which is what most people care about when shooting
ISO doesn’t tell anything about ‘gain’. This is what Tony Northup was trying to talk about in his ‘ISO is fake’ video → ISO means (_veryshort explained in a wrong nutshell, but for this example) that at a certain f-stop and shutter speed, you get a certain exposure in the image. How this is achieved, is completely free. They can manipulate the RAW data how they see fit and think what is best. It is meant as a standard as to what you can expect to appear on your screen on your camera, not what is captured by your sensor.
With some cameras’ behaviour like this can be enabled / disabled by setting certain options in the camera (like 'dynamic range+ modes, however they are called in different manufactures).
Other cameras have two (or more?) native ISOs! Is that dishonest as well to you? At a certain ISO the camera switches to different pixels to read out and drops the gain back down… the result in the screen is what you expect according to your chosen ISO. The result in the RAW is probably blowing you away in a good sense since suddenly all your read-noise is gone. Drop back down in ISO one step and suddenly it’s noisy again. Stuff like this means - once again: now your gear.
In the case of the Fuji example I gave: Nothing dishonest about it. It means that at certain ISOs the sensor gives less shadow-boosting but more highlight headroom. Since every camera at every ISO setting has different behaviour and ‘qualities’, this is no different.
‘Modern shooting’ means exposing for a picture, then dropping the ISO back down to its closest native setting and ignoring the hard underexposure :). The sensor is clean enough to boost back up, and now you get stops more of highlight info to play with.
What Fuji is doing is making this available without having to think about it and make weird exposure settings. Just expose and click :).
‘Normal’ people don’t notice this behaviour because they hardly ever look at the RAW data in a program that does not do the boosting automatically (i.e., does not apply an adjusted base-curve).
If you want to extract every bit of power from your camera, you need to learn what it quirks are. Yes, this means that you can learn that at certain (or most :P) ISOs, it’s probably a better idea to accept a bit of underexposure (since your camera is clear at those settings) while at different settings you need to give it as much light as possible.
What kofa is saying: Modern sensors are ISO-invariant, so they can be boosted up by quite a bit… manufacturers are finally starting to use that fact to give you more headroom. I still think this is a very good thing.
If I was in a more harsh mood, I would just tell people to forget about ETTR unless they had 5+ year old Canon’s. .
Dynamic range is measured from the point at which the camera clips to white (above which there is no usable data), down to the noise floor at which you think the signal-to-noise ratio is unacceptable (note the subjectivity of that limit). Generally speaking, there shouldn’t be such a thing as ‘recoverable highlights’ because once a sensor has clipped, it’s unable to capture any difference in signal. There’s a little margin for error because one channel may clip before the other two, which leaves a little scope for clever algorithms to try to guess an appropriate value for the clipped channel, which leaves the degree of highlight recovery highly dependent on which channel has clipped and what color you’re trying to recover. Significant amount of highlight recovery usually mean that your JPEG is failing to make good use of all the captured data: it’s a failing of the JPEG engine, rather than a strength of the sensor.
However, there are situations in which the GFX 50S will have considerable amount of recoverable highlights, thanks to the way it shoots at high ISO. Above ISO 1600, the camera stops applying any additional amplification and instead leaves a metadata tag telling the Raw converter to brighten the image by between 1 and 5 stops (to generate its ISO 3200 to 51,200 modes). This approach means that, for an ISO 3200 exposure, you’re recording your captured data one stop further down in the Raw file, meaning an additional stop of highlight data is recorded that would have been pushed to clipping, had amplification been used. Your Raw conversion software will probably initially clip this extra data (by applying the default ISO 3200 tone curve), but for these higher ISO settings, you should find a stop of accurately recoverable highlight information for each stop you lift above ISO 1600.
)
) at the camera manufacturer picked a curve they thought looked okay, in line with the name of the Picture Control (or whatever your camera calls it) in which it resides. Same thing with the dt default tone curve for your camera, moreso in that it’s probably based on the camera’s curve. In all the raws I’ve processed, only one or two looked nice “right out of the box”, or in the original linear. A recent PlayRaw comes to mind… All the others required some curving. Right now, I’m experimenting with mixed-light scenes and how much lift is do-able, for instance, to bring an interior close to an exterior window, and those curves can be quite aggressive:
’.
.
.