Scene-referred produces washed out final image (vs display-referred)

[priort (Todd Prior)]

This is what I get… using autoexposure I get +1.5 EV looks okay to me…

Then filmic and the auto picker for white and black…
Nothing more… looks fine and not washed out and dark as the one offered by the OP… I think maybe in that case it was just being too conservative with exposure… Boris’s latest video using the tone ramp really shows a correct exposure…this initially blows out much of the ramp… I am sure many people using scene-referred are maybe just not using enough exposure because they are used to a certain value in the display referred that might be little to no exposure added??

Edit with a small tone eq you can take away some of the shadow and vignette… That may not be good if it is there by design… But basically this is only a third small adjustment

0 Default Image.NEF.xmp (7.7 KB)

[meaningfullyhappy]

Steven, Todd: Thank you for your detailed comments and suggestions. What I’m taking away from your comments:
"All I did was apply the Acros preset in color calibration, then adjusted exposure and the white relative, black relative and contrast sliders in filmic."
I was doing things in reverse: adjusting exposure first, adjusting filmic, and then color calibration last with Acros preset. Could that sequence be the wrong way? It is the sequence Boris consistently has in his YouTube videos and it is different from Steven’s.

not using enough exposure
I agree, I was conservative in amping exposure, which I understand can be a double whammy having underexposed (deliberately) in the first place. Being a bit generous with exposure can actually help and I need to get out of my comfort zone on that.

I appreciate your comments and am grateful to you for your time and suggestions.
Off I go to now play (with more abandon) with scene-referred, exposure, filmic and tone equalizer (and yes, I shall also check out Boris’s tone ramp video and auto-picker for white/black).

[123sg (Steven)]

@meaningfullyhappy

I was doing things in reverse: adjusting exposure first, adjusting filmic, and then color calibration last with Acros preset. Could that sequence be the wrong way? It is the sequence Boris consistently has in his YouTube videos and it is different from Steven’s.

I don’t know to be honest - I was trying to match your first edit, so I found it much easier to compare once it was in b+w… also I usually find I need to (re) adjust exposure after adjusting the contrast. I don’t know if that’s cause I’m not experienced enough or just a preference.
I very often swap around in order anyway! My way is not necessarily the best way - I tend to do what works, but sometimes I get lost anyway.

[MStraeten (MartinSt)]

@meaningfullyhappy

Could that sequence be the wrong way?

the sequence doesn’t matter since darktable processing is done in a fixed order. But you might need to do some readjustments in later modules in the pipe if you change stuff that comes earlier in the pipe… especially when using parametric masks. just trust your eyes

[meaningfullyhappy]

@123sg

(Steven says) I very often swap around in order anyway!

(Martin says): darktable processing is done in a fixed order.

I’m confused…and perhaps it’s because I misunderstood or don’t know enough…
My understanding is:
a) darktable processing is “in a fixed order”, the order being reflected in the listing of modules, with those above following those below, and the modules themselves differ depending on whether scene- or display referred;
b) modules available for scene-referred are unavailable or sometimes inapplicable to display referred. E.g., in scene referred it makes no sense to use Contrast Brightness Saturation or Color Contrast modules nd hence they are unavailable. Instead you use Filmic and Color Balance RGB and Color Calibration etc.
c) Color Calibration always appears before i.e., lower in the list compared to Filmic. So, after applying Filmic if you are to need to use Color Calibration you have to create a new instance of the latter, move it above Filmic and then tweak the new instance of Color Calibration.

is this what Steven meant by “swap around…”? if not what else can swap around mean?
And if, as Martin says, “sequence doesn’t matter”, what can be behind Boris’s creating a new instance of modules lower in the list, moving them above those higher in the list, etc. (as he did with Color Calibration vs Filmic and other such modules in his videos)? if “sequence doesn’t matter” why is the processing in a “fixed order” disallowing modules to be moved up/down the list?

I hope, naive as I seem and am, my questions (and the answers from those that know better) might also help others similarly situated (and I’d like to think I’m not the only one with these Qs!)

[kofa (István Kovács)]

Note that those LUTs require display-referred input (they can only map 0…1 to 0…1). You can rarely (never?) know what input their authors used to match them to the camera’s processing.

The base curve does not maintain the brightness of midtones: it lightens most of the image. Filmic and sigmoid, on the other hand, keep midtones where they are.

Grey steps without any curves:

Note the mid-grey patch in the color picker, showing a reading of 50.

With ‘Nikon-like’ base curve (note how the colour picker jumped from 50 to 94):

Filmic (auto-tuned) shifted the mid-grey to 51.

Sigmoid (defaults) also moved the mid-grey to 51:

Sigmoid with much-increased contrast pushed it to 53:

To roughly match the base curve using filmic, I had to use an exposure correction of 0.42 EV:

Tune filmic:

And set its contrast:

Top: filmic, bottom: base curve:

[meaningfullyhappy]

Istvan, I found your response, and its contents, not merely informative but illuminating.
In short, whether you intended or not, you “taught” me how to fish (with darktable), what to use and how and when, and how they compare to alternatives.
Thank you!

[mbs (Miguel B)]

@meaningfullyhappy

I’m confused…and perhaps it’s because I misunderstood or don’t know enough…

Maybe these notes help:

• As you say, modules are applied to your image from bottom to top, as shown on the list of modules on the right in darkroom. Click on the tab with a power button symbol to see those modules that are active.

• Darktable has a default module order that works well for the vast majority of images. It also has a few different sets of modules that can be activated automatically when an image is loaded into the darkroom (for example scene-referred and display-referred). Click on the hamburger icon to see the available module sets.

• You can turn modules on and off, and make adjustments, in any order you want. Darktable will still run your image through the different modules from bottom to top.

• It is usually more efficient to make adjustments roughly from bottom to top. The reason is that, if you make a change near the top, then another change near the bottom, you often have to go back and readjust the near-the-top module. This back-and-forth is minimized if you work bottom to top.

• This approach may require some getting used to. For example, adjusting exposure may blow your highlights, but it doesn’t matter, because filmic or sigmoid will fix that. Likewise, filmic will often produce washed-out colors, but it doesn’t matter, because you’ll fix that in Color Balance RGB module.

• The reason display-referred modules should not be used before filmic or sigmod is that they use a different scale. Display-referred modules want all pixels to be in the range [0, 1], but scene-referred modules use the scale [0, \infty].

• You can move modules around to change their order in the pixel pipe. My recommendation is to avoid doing this until you master the default order.

[meaningfullyhappy]

@mbs

Likewise, filmic will often produce washed-out colors, but it doesn’t matter, because you’ll fix that in Color Balance RGB module.

Miguel, Color Balance RGB is below filmic in the pixel pipe.
So if it is to fix the washed-out colors produced by filmic does it explain why I would need to create a new instance of Color Balance RGB, move it above filmic, and do the “fixing” in that new instance, as said fixing would follow and not be overwritten by filmic?

[mbs (Miguel B)]

Yeah, that’s what I meant by “making adjustments roughly from bottom to top”. In other words, there are exceptions, and this is one.

Filmic is the last of the scene-referred modules because its output is mapped to the range [0,1]. But it is not necessarily the last module to adjust.

I usually start by adjusting exposure, then adjust filmic. The image will often require contrast and color adjusting, often done with Color Balance RGB and/or Tone Equalizer, left in their usual place before filmic. Sometimes (not often) filmic may need a final tweaking.

[meaningfullyhappy]

Got it, very useful and practical, thank you, Miguel.

[123sg (Steven)]

@meaningfullyhappy

Boris’s creating a new instance of modules lower in the list, moving them above those higher in the list

I think the others have probably answered your questions
But yes, when I was talking about ‘swapping around’ I meant the order of how I adjust things, which doesn’t affect the order of operations in darktable internally.

BTW, it’s rare that I need to move modules in the pipeline - it can be useful, but to use your example about adjusting colours in conjunction with filmic, one can usually happily leave the color balance module in it’s normal position. So it feeds its adjustments on to filmic, which in the normal way is meant to be at the end (or near the end) of the pipeline. Because it does the transform from the scene-referred space into the display space… anyway, the default module order is quite carefully thought out fo best performance, so one only needs to move them if one has a specific point in mind, in terms of some operation that needs to be done before/after something else.
But all this is not affected at all by the order that you adjust sliders - you can jump around to your hearts content!

[priort (Todd Prior)]

@meaningfullyhappy

I agree, I was conservative in amping exposure, which I understand can be a double whammy having underexposed (deliberately) in the first place. Being a bit generous with exposure can actually help and I need to get out of my comfort zone on that.

For the exposure thing the demonstration with the tone ramp in Boris’s latest video really demonstrates it well … ie the part where he draws the face over where middle gray needs to be and then just adds exposure to get that shift…much of the ramp is then overexposed until remapped with filmic or sigmoid… I think this sometimes gets missed when people move to doing scene referred edits and I think they can be conservative as often you could be adding 2 or even more EV. This would seem strange in other software that has opened the image with a profile that has a tone curve and or other adjustments out of the gate…

[priort (Todd Prior)]

@meaningfullyhappy

Miguel, Color Balance RGB is below filmic in the pixel pipe.
So if it is to fix the washed-out colors produced by filmic does it explain why I would need to create a new instance of Color Balance RGB, move it above filmic, and do the “fixing” in that new instance, as said fixing would follow and not be overwritten by filmic?

If you notice when Boris does this he has made targeted corrections to the image and has it where he wants it Then for the next step he is sometimes trying to impact the color from that exact point that he has developed so he moves the module so that the effect will be added from that point instead of the default position where it might impact the work that he has done. He knows very well what he is doing and of course we can all learn from those techniques he presents but for the basic workflow and based on single instances there is the prescribed module order as people have mentioned already… There can also be the nuance as it applies to masking and what gets used as the input along the pipeline… Inserting extra instance in the default position which is earlier than some of the corrections you have already made could impact parametric masks that you created in those modules…

[s7habo (Boris Hajdukovic)]

I see you have already received very good answers to your questions.

I’ll briefly clarify why I changed the order of the modules in Episode 50 when converting to black and white, which is causing the confusion.

First, let’s clarify how the conversion to black and white works in the color calibration module.

First we notice that in the face the red color channel dominates and in the scarf the blue one:

When I move the slider in the “gray” tab of the color calibration module, the corresponding color channel becomes brighter. For example, if I increase the red color channel, the face becomes lighter because red color dominates there and the scarf not so much:

But the problem is, the face looks very flat, loses the details.

If I now instead of red, take the blue color channel, scarf becomes brighter and the face not so much:

Also the skin details in the face are amplified, because there the blue color channel is lowest and they remain even darker than the face.

Now that I know that, I can decide how I want to do the transformation.

For example, I want to have the face lighter but in such a way that I don’t lose the skin details too much. So, I combine both color channels according to my taste.

This way I find a good balance according to my taste:

And as a last step, I want to increase the contrasts a little bit in the photo as a whole. I can do that with color balance rgb module, which is located in the order above color calibration module. That means I increase the contrasts after conversion to black and white.

For this I will use only shadows and highlights slider in “perceptual brilliance grading” function in color balance rgb module:

And now I have a good black and white version of the photo with good contrasts.

Now, let’s say I want to have a version just like this one, but with a slightly more darker scarf, so that I get a better “framing” of the face.

Since in the conversion example above I used the combination of mainly blue color channel with a little red color channel to get optimal face brightness and details in the face, scarf would be darker if it was red instead of blue.

That means now, before I make the conversion to black and white, I have to make the scarf red to get the desired effect - darker scarf later in the black and white version.

It also means that I need an instance of the color balance rgb module before the color calibration module (before the conversion) to be able to make the color shift of the scarf from blue to red. So, I move the color balance module under the color calibration module:

In the color balance rgb module I mask the scarf…

…and change its color in the “4 ways” tab:

I also change his name to “Color of the scarf”.

Now I use the color calibration module as in the example above to make black and white conversion:

At the end, I need another instance of the color balance module to increase the contrasts in the photo after the conversion.

I create new instance, rename it as “contrast” and move it over color calibration module with which I did the conversion:

Now I can increase the contrasts just like in the first example above:

And as you can see, now I have similar results as with the first conversion, but with a much darker scarf.

[meaningfullyhappy]

Boris, Todd, Steven, Miguel and others who commented/responded to my Qs:

My gratitude and appreciation for your help, time.
I believe I now understand darktable, filmic, the pixel pipe and flow, what to do and when and why…far better than by reading the manual, playing with the app, etc. That your comments were pertained to and were readily applicable to an image I had taken made them even more striking and edifying.

I believe I have enough to put to work what I’ve learned (at least until I hit the next road block which I hope is some time off into the future!) I’ve already updated to 4.2, have sigmoid and keen to use it after review of Boris’s new tone ramp video

Thank you very much!

[priort (Todd Prior)]

Now you know why I said Boris moves things around because he knows what he is doing His videos on white balancing with the channel mixer and the ones he does on bw conversions are really insightful even just from the point of looking at the photo, analysing it and then deciding where to go from there…

[Redbrook (Mustafa K.)]

Hi Boris,

this is a very illustrative and compact lecture on B&W conversion in Darktable. Thank you!

[Redbrook (Mustafa K.)]

I have a question, just to confirm:

You say you used “4 ways” of “color balance rgb” module to change the color of the scarf from blue to red (see the concerning photo). I understand that you keep the hue at the starting position that is red and you move the chroma sliders of “shadow lift”, “highlights gain” and “power” to 100%.

[meaningfullyhappy]

Boris, now that I’ve had time (and some footing in DT mechanics) to better understand and absorb what you detailed in your example above I have a few Qs and would appreciate your clarification on them.

I noticed you use the waveform scope. Following Todd’s suggestion I started using it too and find it more useful than a histogram (which was a giant step forward to not having anything, which was the norm before digital SLRs!)

You say: “red channel predominates (in the face)”…“increasing the red channel in color calibration the face becomes lighter because red color dominates there…”, “blue brightens the scarf, not the face…” and finally “find a good balance (combining both channels) according to taste”.

I’m curious how you used, if you did, or would use the waveform (in one of its three incarnations) in making those choices and decisions. i.e., beside the eye alone, what in the waveform can we use to guide our choices and decisions?

In your video Episode 50 (Conversion to B&W) (Darktable Episode 50: black and white conversion part 3 - YouTube) you first prep/fix the color image and get it ready to convert to B&W. I find this a very valuable step. The waveform is in color too, perhaps suggesting how it can be used to answer what I’m trying to get at.

In your example (in this thread) you had already converted the image to gray scale and then explain how R or/and B help.

In either or both cases I’m curious how the waveform–whether in combo or RGB Parade–can be used to guide in the conversion and production of a stellar B&W image.