Raw black/white point module - Do I have a problem?

This is from the manual, which has to be read first, I know : ( .

This module is activated automatically for raw images and makes sure that the camera specific black and white points are taken into account. Default settings are applied for all supported cameras. Changes to the defaults are normally not required.

I guess it is more likely that some of my files are faulty for some reason.

For both of my Pentax cameras (K-5 and K-3) where I use DNG the white point values differ from image to image. So the values are definitely read from the file.

Maybe the GR III has some nifty features we don’t know?

Hi! Has this been sorted out for you? I just noticed today thay bunch of my pics done recently are washed out of colours in Darktable when I switch from lighttable to darktable module. I loose nice pic and get dark, gloomy one. First time to see that! It is so frustrating

A raw file needs some processing to give a visible image. If you do only the absolute minimum, you’ll end up with an image in scene-referred color space, (i.e. pixel values are linear with light intensity). Such images will look too dark and flat/washed out.

Have a look at the history tab (darkroom mode, left of the image) and check if either filmic or basecurve are active. If neither is present, that’s the cause of the dark pictures.

You still get a nice picture in lighttable mode, as dt uses the jpeg embedded in the raw there. (It’s a bit more complicated in practice , but that should be explained in the manual)

2 Likes

I will look into that, thanks. I use Darktable programme and I found that strange as I was always seeing the initially processed pictures in both views (this nice jpeg view with bright colours) in lighttable and darktable mode. I read that maybe there is a trouble with data set by camera on my new pics connected with “raw black/white point”? To check it I opened some old RAW files that I already worked on and suddenly they got washed out as well even though they were nicely processed by me. I do not recognize that specific tool was used before by DT itself.

First get an idea on a darktable workflow - darktable is not Adobe lightroom.
There’s a wip documentation, but already very useful: https://elstoc.github.io/dtdocs/overview/workflow/
Even for a beginner with darktable i recommend the display-referred workflow. If enabled it gives decent initial results, which can be improved with just a few modules.
quite old but still valid: https://discuss.pixls.us/t/darktable-3-0-for-dummies-in-3-modules

(But this is off topic in this thread)

I am using DT for few years now and that is first time I see the difference between lighttable and darktable modes. It was always setting some changes so the pictures looked fine without any of my further steps. I thought it is connected with some latest releases of DT as it changea even my old pictures which were exported tp jpeg

Can you maybe restate your problem ? I have the feeling people are not getting the issue.
You have old DT processed files (jpeg and raw ) and when you open them with newer versions, the rendering is not the same ?

Thanks Olivier. Yep, I had 3.0.x version on one computer and last month I worked with my raw files which looked same as on camera, in lighttable and darktable modes, I just did some slight changes to raws and got decent jpegs. Yesterday I opened DT 3.2.1, the newest one I believe, with some new raws from last week. I saw nice pics - same as on camera - in lighttable and when I opened them in darktable mode suddenly those became greyish amd dark. That never happened to me. Now I would need to do plenty of changes. I thought it might be some issue with option “raw black/white point module” which was not picked by DT before, in my opinion. It is first thing that DT changes for RAW now, before it was not mentioned on the history panel at all. That cannot be switched off though. And apparently it ruins my old pictures that I processed even year ago. They were looking neat and I exported them to jpeg. Now they started to look terrible in DT. So it must be sth with DT or setup as it has an influence on all of my pics. Maybe I should tick off sth in DT?

This sound like the base curve resp your edit is not applied automatically. Maybe the sidecar files are not recognised. Btw: Greyish and dark is what the real raws actually look like.

I know that raw images are not impressing ones :slight_smile: but DT is always applying lots of modules so they are easy to curve in afterwards. And now it seems that sth happens with colours just straight from the beginning. On the history tab there is always each step mentioned starting from 0.original - which is grey and raw raw and then some steps done by app. Now I noticed that 1. Raw black/white appears and that was never shown before. That messes with my pics, they do not get better at any stage later on.

I got differing values for raw b/w points even when I shoot in manual mode and use same settings for several images, panoramas for example.

There have been some changes in recent darktable master that copies raw b/w points using copy/paste on history stack. Since then I have to use the “copy parts…” button and unclick the raw b/w point when I want to copy a whole history stack.

How do you do that? I was always copying whole history and applying it to another picture, no clue on how to pick just needed changes? Were you creating a style from the history stack?

I lighttable, history stack, you have 2 buttons: “copy…” and “copy all”.
And when pasting, you have the same kind “paste…” and “paste all”
With the simple version (“copy…” and “paste …”), you can select which modules you want to copy or paste.

1 Like

No: raw black/white point has always been there, it’s an essential step. But, as it’s an automatically applied, necessary module, it was not part of the history – that has changed.

When you import raw photos, darktable, by default, will show the JPG preview (embedded in the raw even if you don’t shoot raw + JPG) on the lighttable.

When you open the raw, darktable will apply a bunch of modules. Certain modules will always be applied (e.g. demosaic, white balance, raw black/white point, input profile, output profile). With 3.2.1, you have to set which other modules you want applied. By changing the workflow type in the settings (https://elstoc.github.io/dtdocs/preferences-settings/processing/ - see under ‘auto-apply pixel workflow defaults’), you can either enable the basecurve-based, legacy ‘display referred’ workflow and the corresponding modules, or the now recommended ‘scene referred’ workflow, which will enable exposure (with a setting that automatically corrects in-camera exposure bias + adds a small additional amount so the end result is about as bright as what the camera would give you), and also filmic (to restore highlights blown by the exposure adjustment and apply a contrast + saturation boost).

If that setting is set to ‘none’, you won’t have any kind of ‘boosting curve’, and are left with a flat-looking image.

Hi Kofa, thanks for answering! Yep, I can see flat raw when I pick 0. Original item on the history list, then I pick 1. White/Black point and pictures get very, very dark. Then DT tries with other automatic tools and it doesn’t get much better. It happens for both Processing setups: Scene and Display referred. If that White/Black point was always there but just not shown, why does it blacken my photos now? Maybe I should pick None - to see the difference…

If, due to a bug, raw black / white point set a very high white point, it’d darken the image.
Can you share a raw file somewhere? (Dropbox, Google Drive shared folder etc.)

If you check the module, what values does it set?
With my 12-bit Panasonic LX-7 (theoretical maximum: 4096):
image
With the 14-bit Nikon D7000 (theoretical maximum: 16384):
image

Now I set Processing to Scene referred so the darkness is not that dramatical at stage 1. White/Black point, but it gets much worse on the Exposure level now. You can see both from printshots. Looks like something is wrong with this new feature for me… According to my camera Nikon D7200 - pictures are 14-bit.

OK, you can see the solution in the exposure module:
There’s a checkbox, saying ‘compensate camera exposure (+1EV)’. And then an exposure of +0.5EV.

  • The first of those means that when you took the shot, you applied an exposure compensation of +1EV in camera (that is, you tried to make it 1 EV brighter than the camera would have used). This means that the module applies -1EV to compensate, making the image darker.
  • The second setting means that after compensating for the exposure correction / bias used when taking the shot, +0.5 EV is added.

The end result is that, in your case, the exposure module is making the shot 0.5 EV darker.

So don’t panic, there’s nothing wrong. Just adjust the exposure manually as you see fit.

The exposure module is trying to compensate for the exposure correction dialled in the camera because people often dial a negative value there to avoid clipped highlights (making the image darker in camera, saving the highlights, with image brightness to be corrected in post-processing). However, filmic requires that you start with a photo that’s exposed properly for midtones (filmic is centred around midtones). Don’t let it fool you that ‘exposure’ comes after ‘filmic’ in the history: the processing order is shown on the first of the tabs on the modules block, under the histogram: exposure comes first, then filmic (the ‘pipeline order’ lists the first-to-be-applied module at the bottom, with the last-to-be-processed at the top).

So: raise the exposure, not caring about the highlights; then go to filmic, and raise the white level there to bring them back.
Please read this article (long, but extremely useful and important): darktable's filmic FAQ

1 Like

Black and white point settings look about normal for a 14-bit raw file: max. value for whitepoint would be 2^14 in your case, or 16384, in practice it’s a bit lower.

Have you tried to follow the normal process of setting the exposure to get the middle grays (in this case, the foreground grass) in the proper range, and then use filmic to tame the shadows and highlights (i.e. get them within the range of your display). If the history you show is the auto-applied one, I’m not particularly surprised by the result, you will need to adjust at least exposure and filmic (and you shouldn’t need to touch the raw black/white point here)

Also, is the bright region top centre still within the sensor range, or clipped? Dont’ forget that such a bright region will force your camera exposure down, leading to a rather dark foreground