Tone Equalizer - How to raise shadows.

(This is a new topic started at: [DE] darktable.info – A new resource for the German-speaking community (Modern Workflow & AgX) - #143 by paperdigits)

Let me start to explain where I am coming from. I have been using Lightroom and Capture One for years before moving to darktable. But I was getting very annoyed by the vendor locking. And I am preparing a move to Linux, and well darktable was by far the best option I tried (because the interface is clean - surprise surprise)

This post is intended to explain my learning process with the tone equalizer. Nb. I have used the current version of the tone equalizer, the UI is better than the previous one, but for my story here there is not really a difference.

So as example to what I struggling with. This is the image as shot. Yes, it is under exposed, but that is rather as intended. So ensure the highlights are not blow out, which is extremely easy to do in snow. Please note, while this is stationary shot, birds move a lot, which doesn’t allow much time to and carefully expose.

So in Capture there is a module called ‘High Dynamic Range’ which is very easy to use. For the image I raised the exposure a bit, bumped up the shadows are gave some punch using the highlights. This is literally 30 seconds. (of course not much control)

So this is wanted I to replicate in darktable. So as I said, I started to use the shadows and highlights module but that will often create halo’s or blur for me. So I moved over to the Tone Equalizer, as the documentation suggests. The first thing to wrap your head around is that whole maks thing. Note, coming from a tool like Capture One that whole mask thing (which I assume that they use as well) is cumbersome, why not just one or two sliders??

So the first thing I read in the manual:

When used together with filmic rgb, this module replaces the need for other tone-mapping modules such as the base curve, shadows and highlights, tone curve and zone system (deprecated) modules.

Okay… But I am using sigmoid. Does that means that I can’t use this? Well I figured that sigmoid and filmic rgb are both tone mappers so just let’s try.

The next section in the manual describes how tone equalizer works. It reads well, no comment here. The only things, if you want to know, how to use the module then it is odd add first. But it makes sense that it is there at the beginning.

Then I read this

Examine your image beforehand to identify which regions you wish to dodge and burn, and use the controls on the masking tab to ensure that those areas are reasonably separated in tone within the final mask. This will allow those regions to be adjusted independently

This gave me the impression that the regions are independently. In my mind that means that if I change the slider for one region, other regions are not affected. But given how the curve is changed, this is not the case?? Or what am I missing here.

So let’s open the image in darktable, and measure how dark the duck is. That is -2.5EV

So let’s raise the real dark shadows (-6EV).

At the beginning I was baffled by the result… What is going on here? I pushed -6EV and evyerthing is strange… So are they independent or not??

As a beginning user of darktable at this point I was completely lost. I actually considered removing darktable, because if a program can’t handle shadows properly… it is a lost case for me. But this seemed user error to me (and I figured rather quickly that darktable is not holding your hand like Capture one is doing). So let’s try the advanced tabs.

But first is read:

If the mask’s histogram is evenly spread over the entire tonal range, sliders towards the top generally affect the shadows, whereas sliders towards the bottom generally affect the highlights. You can check the spread of the histogram within the advanced tab.

From a mathematics background I know what the spread of a histogram is. But I hat to read this sentence 3 times before I got an idea what it could mean. This would be more clear in my opinion

When the mask’s histogram spans the entire tonal range of the image, the upper sliders mainly influence shadow tones of the image, and the lower sliders mainly influence highlight tones. To check the histogram, open the Advanced tab.

Next I read:

If the histogram is too bunched up, this means your mask doesn’t have a good spread of intensity levels, which makes it harder to independently control the brightness of the different parts of your image.

As a non-native speak I was puzzled by too bunched up, I actually had to look this up in a dictionary…

Moving a single control point will also affect control points on either side to ensure the curve remains smooth. This behaviour can be adjusted using the curve smoothing control.

So the regions are not independent??

curve smoothing
Control how the curve is interpolated between control points… Move the slider to the left for a more well-behaved curve, but beware that this can result in harsher tonal transitions that may damage local contrast.

I am sorry, but I have no idea what a well-behaved curve is. Initially I thought that it would fix the thing that I broke earlier. But clearly that is not the case :slight_smile:

Okay let’s continue reading.

mask exposure compensation
Adjust the mask’s histogram to the left or right. If you have used the exposure module to adjust the image brightness, you may need to offset that adjustment by using this slider to re-centre the mask’s histogram. Click on the wand icon to the right of the slider to set the exposure compensation such that the average of the mask’s histogram will coincide with the central –4EV control point. The slider can then be fine-tuned as required.

mask contrast compensation
Dilate (spread out) or compress the mask’s histogram. The wand icon to the right of the slider will propose a reasonable starting point, which can then be fine-tuned to optimize the spread of the histogram under the tone equalizer control points.

Okay the mask exposure compensation is clear. But I was baffled when I was trying the mask contrast compensation slider. I would assume that the peak of histogram of the mask says in it’s place, but would get lower. But sometimes it can jump 1EV ??? I would not have guessed this from what I read.

From all want I read so far, I find it (still) very hard to go into the tone equalizer and correctly predict what changing the sliders of the mask will have for effect on the image. I mostly do something by trail and error, and it works. And most often, I am happy with the result.

But I am not always happy with the result. Sometimes I just can’t predict where I want the shadows to be at. And I noticed that raised shadows seems noisier then I was used to in other software.

Note: I have not covered the masking tab section of the manual. That is above my pay grade at this point

3 Likes

I’m not sure what your intent is. If you want to learn how to process this image faster/more efficient or just how to use the TE module? It seems like you are trying both with your reference to Capture One. If you want, you can share this image as a PlayRaw and folks can share their edits/steps.

I’m assuming you just want a quick good results first (aka 30s Capture One style). The main process is described in the process section of the manual. 1) Adjust exposure in the Exposure Module to get the mid-gray point right. 2) Go to the tone mapper (filmic or Agx) and adjust the white (I think this is what Capture1 calls highlights) and black points (shadows?). Then adjust contract, color, crop. After that, if you want to make the female duck brighter relative to the rest, then go to the TE module. There is an excellent video from AP in YouTube that will explain this far better than me typing here.

1 Like

Coming from a previous discussion, my intent is to show with the section of the manual I struggling with. And of course I a very open for a better editing technique.

Do you have a link? Who do you mean with AP?

1 Like

In rereading the Tone Eq reference a few hours ago, I also noted the same thing. So this will be adjusted. It is from a time where the available tone mappers were either base curve or filmic.

I think what you’ve missed in the whole thing is that you never adjusted your mask, as you can see in your only screenshot of the module, your histogram is indeed bunched up at the left hand side of the module.

If you click the button for mask visualization, you can see the current mask. Alternatively, you can use the preset for “simple tone curve” and that will get rid of the masking part. That makes things more straight forward to use, but you really loose a lot of the power.

If you click the two eye dropper buttons for Mask Exposure Compensation and Mask Contrast Compensation, you should get a pretty good mask for your image. This what I do 98% of the time.

I have been for sometime contemplating putting a very short “usage” paragraph at the top of each module reference. For Tone Eq, it’d probably read something like:

Adjust the Mask Exposure and Mask Contrast Compensation sliders or click the auto picker icons, then use the simple or advanced tabs to adjust tones in a given region. Note that if you are not able to manipulate the expected tone, you should use the Mask Visualization button to adjust the mask, or turn off the mask all together by using the Simple Tone Curve preset.

What do you think of that?

Another thing that made aide you greatly is doing a Play Raw of an image you’re struggling with. Describe your issue and people will give you XMPs you can examine and you can ask people how they’ve done things when you like a result.

3 Likes

This was a pretty good thread…@kofa has a nice mask demonstrated in this thread…

AP’s videos I think are all pulled

Boris has a few video and I believe one in particular has a reasonably detailed explanation of the mask creation and manipulation

2 Likes

Great!

This works better.

Of course in my last picture, I did adjust the mask. And I am using those pickers now, als about 98% of the time or so… But for me the ‘simple’ tab at first did imply that was the mask was taken care of. Actually I don’t look that the simple tab nowadays at all…

Thanks for that video! I have not watched all episodes yet… thanks will check it out!

2 Likes

Resulting: Remove mention of filmic from tone equalizer's description by paperdigits · Pull Request #805 · darktable-org/dtdocs · GitHub

5 Likes

Can we see more photos of the new ui that you came up with?

What?

There was an effort to rework the Tone Eq UI but that is not in the current release…the mask exposure and comp sliders are under the mask now instead of in the masking tab but that is about it…

Another tip, @martinus, is to put the tone equalizer module instances below exposure. That way you don’t have to adjust the “mask exposure compensation” of every tone equalizer instance every time you make adjustments to the global exposure.

If you have 7 tone equalizer instances active (like I sometimes have), then you have to go back and adjust them all if you change the exposure. I don’t think I have to add how much of a pain that can be. :smiley:

I should probably have some lua script (?) to always put the the tone equalizer below exposure, because that’s how I use it.

2 Likes

Simply arrange them in the order you want and save it as a preset. Then you can assign a hotkey to the preset.

2 Likes

There has been many helpful replies in this thread already, but I don’t think it’s been mentioned yet how useful the presets are…

For dynamic range compression like in your example above, I almost always start out with one of the çompress shadows/highlights presets.


They set up the mask (including very importantly the detail preservation) in a way that most of the time works really well.
Usually all I need to do is tweak these two:
image
to make sure the curve is covering the brightness range I want to work on, then I usually double-click the curve to reset, before finally dragging the curve (or moving the cursor over the image being edited and scrolling) adjust the shadows/highlights/whatever.

I have to agree that for the kind of context you are using the tone eq for (that goes for me too) the simple tab is not very useful. It’s another of those cases where darktable gives the whole toolbox of possible approaches, instead only the specific tools most commonly wanted… as a home mechanic I rather like this, but it can be confusing initially.

My other question for @martinus is, how is the page on darktable.info better? Tone Equalizer - Mastering DT 5.4: Workflow, Shortcuts, Themes & AgX

I think that page gives wrong advice, to be honest. In particular:

If it looks weird (Halos):
Go to the “Masking” tab.

  • Smoothing diameter: Controls how soft the mask is.
    • Larger value: Softer transitions, less local contrast, but less risk of artifacts.
    • Smaller value: Preserves more local contrast (“punchier”), but risk of halos increases.

A smoother (‘softer’) mask preserves more local contrast (neighbouring pixels are treated similarly, their darker/brighter relationship is preserved), but can lead to halos; a mask with more abrupt transitions is less prone to halos, but, since neighbouring pixels within an area may be assigned to different ‘zones’ in the mask, can lead to a loss of local contrast.
Additionally, edge refinement is not mentioned at all.

1 Like

I find the shadow and highlights module still has a use and can be used in combination with or instead of the tone equalizer. I see these modules are distinctly different even though they do a similar function. Just because shadow and highlights is a displayed referred module doesn’t negate its usefulness in my workflow.

Also with tone equalizer it is worth experimenting with the preserve details option and seeing how that affects the adjustments you have made in tone equalizer. I find often but not always one of the guided filter options looks better than the default EIGF.

Well, the article you link is better in the sense that it presents a workflow that can directly be used (the how to use it). It is more guided (at least for me) than what the official manual is presenting. If I would have had this article when I started learning dt, it would probably have saved me the frustration of the strange behaviour I saw at first.

It is rather opinionated in this:

  • Forget the sliders – use the interactive cursor and the mouse wheel directly in the image.

But in my context this would have actually not been bad advice.

I would not say that this article is perfect of course. @kofa makes a valid point here.

But that is edge refinement is not mentioned, I don’t see why that is a problem for the scope of this article. But of course, the article is far, far from complete.

But in general what I am struggling with in the manual, is how to extract a workflow for a module. The user manual doesn’t tell me in which sequence to use things (I think because it doesn’t matter much - until it does). And often not much in the terms of problem solving.

Actually, the first question of the faq. on the website: faq | darktable “Is darktable a free Lightroom alternative?” explains that rather well: there is no default workflow and that the learning curve is steep.

But for me it would have been helpful if a suggested workflow would have been presented (like the linked article) in the manual. Or that the manual would link to more ‘hands-on’ article.

I hope that this makes sense…

Yes, this. In general, I would argue that the manual should have a more hierarchical structure that allows one to read incrementally larger parts of the text only as needed.

In order of presentation and content length/density:

  1. one short paragraph explaining in very layman terms what a module does (and, when meaningful, establishing a parallel with more or less equivalent tools in other software).
  2. one (or a few) very short examples of a couple of paragraphs each (with before/after images) showcasing a common workflow and what one can reasonably expect from the module.
  3. one section about recommendations, best practices and common pitfalls.
  4. [possibly optional, depending on how verbose the next section is] one brief overview of all the controls, with one or two sentences each describing what they are meant for.
  5. extensive documentation of each and every control and all the technicalities.

I tried a similar approach with my attempt to refactor the shortcuts documentation, even though in that case it is not a module that we are talking about, so the exact same sequence does not apply.

If there is consensus that this is desirable I would be more than happy to contribute to the effort.

4 Likes

This sounds really good!