Blacks out-of-gamut

I was trying out geeqie because of another thread I was reading, and I came across some old raws from a drive location I had paid no attention to. I started working on this one in dt 3.4.1. I have not finished it, but I became stumped when Filmic RGB turned it back to a very dark image. I discovered that (I think) that is because there are out-of-gamut blacks. So I tried to get them back with the Filmic black and white relative exposure controls. Am I on the right track? Or, is there some generally accepted methodology for dealing with this?

DSC_0018.NEF.xmp (9.0 KB)

DSC_0018.NEF (22.1 MB)

This work by Tim Cuthbertson is licensed under CC BY-SA 4.0 [image][image][image]

4 Likes

Do you have the original…how was the exposure before you applied filmic…

Sorry, I will edit to add the original. The exposure was +3.3 EV.

Here is my version


DSC_0018_01.NEF.xmp (9.4 KB)

2 Likes

@Tim

The reason you ran into this way-to-dark problem is because darktable isn’t all that smart when applying auto scene referred settings. Especially in this case because it could be picked up easily and resolved by darktable while parsing this RAW.

The actual problem is this: You use scene-referred, which activates and sets filmic and exposure. This does not work correctly when you shoot manual and, for whatever reason, have adjusted the exposure compensation button on your camera. darktable blindly add/subtracts the found value (+3.3 in your case), which it shouldn’t do when you use M(anual) mode because in that case is doesn’t change any of the ISO, speed or aperture settings.

It is one of the reason why I’ve set my workflow to none and do it all manually. I always shoot in manual mode, dial-in my settings using the (live) histogram. I then use the exposure compensation button to change the lightmeter setting (which is the only thing the exposure meter changes in Manual mode) to the exact middle so I can have an easy indicator that I need to adjust my settings by 1/3 or 1/2 stop or redo the whole dial-in process if there is a larger difference.

Anyway: Nice dogs!


DSC_0018.NEF.xmp (8.4 KB) darktable 3.5.0+2096~g629a0481d

5 Likes

my version

DSC_0018.NEF.xmp (8.0 KB)

cheers

2 Likes

Cute dogs!
With dt.


DSC_0018.NEF.xmp (9.8 KB)

1 Like

Thank you for the play. My take in RT DSC_0018.tif.out.pp3 (13.6 KB) and thereafter a little, little bit G’MIC.
I’m still learning (RT/darktable).

1 Like

Strange effect, @Tim. Couldn’t believe my eyes. Thanks for sharing and thanks to @Jade_NL for this detailed explanation.

Cute dogs they are !

dt 3.5.0 filmic

DSC_0018_01

DSC_0018.NEF.xmp (12.9 KB)

dt 3.5.0 sigmoid

DSC_0018_01.NEF.xmp (9.4 KB)

1 Like

I start my editing session by backing out of the pre-applied stuff, including highlights adjustment. Then, I do exposure, color calibration, and filmic. Then other modules as I finish the ways I want things to look. So, at least in my mind, I am bypassing most of that. Maybe I’m fooling myself.

For this image, all I had before I started at White balance was Original, Orientation, Raw black/white point, Demosaic, Input color profile, Output Color Profile, and Display encoding.

Thanks, and I will try to digest your advice.

Interessting effect. I personally don’t care so much, because in general I more or less start from “scratch” (original).


DSC_0018.NEF.xmp (13.0 KB)

I like the eye colors!

3 Likes

Same here.

I kinda stopped using darktable for my personal stuff but I do tend to start as bare-bones as possible, no matter the RAW editor I use. Those auto-applied “goodies” come with their own caveats, I rather mess things up myself :laughing:

1 Like

I think it also depends on how many images you want to edit. If you have hundreds of shots from a wedding, an automatic start might come in handy :wink: .
However, in general want to see (and feel) the raw before I decide what to do with it. So I try to start with as few operations as possible (demosaic is okay though :sunglasses: ).

1 Like

First, a bit of terminology: There’s no gamut in ‘black’. Black is the absence of light energy and gamut is about the ability of rendition media to resolve colors of extreme energy, at the other end of the energy continuum, so to speak. What you’re probably concerned with are negative values, which are the product of manipulation.

The data right out of your image has no negative values; indeed, the minimum value of any channel is about 147. The interesting thing I found is in the NEF metadata, the MakerNote tag BlackLevel contains four values, all 600. If you subtract that from each pixel, you drive most of the meaningful data in the image below 0, given that the maximum value for this camera with it’s 12-bit sensor is 4095. Libraw delivers a more sensible value, 150, which contributes to a “more colorimetric” image.

Assuming negative values are clipped to 0 by black subtraction, I found a few subsequent operations that would drive values negative: some demosaic algorithms, lanczos resize, and convolution sharpen. The all affected only an insignificant number of pixels, my export conversion clips all such to 0 and the image looks fine.

Also, this image responded to a rather tame filmic curve. I can’t comment about the darktable filmic; I started with a simple normalization of the linear data to 0.0-1.0, and I only had to boost the B param of rawproc filmic to 26.2 (from 16.2) to get this, in rawproc:

I’m posting the 800x600 proof to demonstrated the value of rendition resizing in reducing noise…

This is a really good demonstration image for all of the raw-processing fundamentals; with your blessing I’d like to use it in a demonstration video…

6 Likes

I know I am not answering the original question, but here is my take using RT Dev.

DSC_0018-2.jpg.out.pp3 (30.4 KB)

3 Likes

Thank you Glenn. While much of that is still over my head, that is the sorts of answers I was asking for. I am more trying to learn what was wrong, in general, and how to deal with it, in general, than how to edit this specific image.

1 Like

So, I just tried this:

I took the history all the way back to “original” and compressed the history stack. This gives that there are no out-of-gamut pixels, and a very dark image (expected).

But the pipeline on the right still shows several modules as active. The one lowest in the stack is “raw black/white point”. darkroom does not allow me to deactivate this module. And when I click on it, the out-of-gamut pixels are highlighted.

All four black levels are shown as 150 in the raw black/white point module. White is shown as 3880. So, I think you may be suggesting to change those 150s to something else. I will play with it and see what happens.

PS - If I change them all to 141, there are about three highlighted out-of-gamut pixels. If I change them to 140, there are none.

And here is my “final” (heh heh) attempt based on the above post, with the raw black points all set to 140. I don’t know why the numeric suffixes on the files differ, but the files were created at the same time.

DSC_0018_02.NEF.xmp (8.5 KB)

1 Like

Just so you know:

There are a handful of modules (5) which cannot be disabled. Those are the bare minimum needed to go from a bucket of bits (the RAW file) to a very basic “viewable” image.

1 Like

darktable 3.4.1.1

DSC_0018.NEF.xmp (14,5 Ko)

1 Like