Experiments with a scene-referred local contrast module - proof of concept

My UX issue with contrast eq (or the RawTherapee module you mentioned) is translating the x-axis of that curve (or the slider index etc) to the scale it operates at on the image.

The manual recommends visualizing it with the difference blend mode, which is feasible, but not a smooth user experience. I have used that module a lot too, but still don’t have a good intuition about this.

But have to be moved all separately.
If it’s just two sliders, fine if there are six like in RT…

That’s why I wrote this:

but how often do you need this on more than two levels?

1 Like

Nice example, thanks for sharing! Very informative

2 Likes

Hello,

Apologies for my absence from the discussions.

If I understand correctly, some of the contributors are in favor of keeping the module as it was very well-designed by @Wilecoyote

As its author is also in favor of keeping it as it is, I don’t want to impose my proposal.

On the other hand, this exercise has given me another idea, that of a ‘pyramidal contrast’ type module that could very well find its place alongside this one.

EDIT : Sorry, I wrote too quickly. I hadn’t read this comment, which I find very interesting and thought-provoking.

I hope to be able to come back with a proposal over the weekend.

Greetings from Lubéron,

Christian

2 Likes

Hello,

Thank you very much for your feedback.
And well done on this magnificent photo.

Greetings from Lubéron,
Christian

1 Like

Two levels are sufficient, of course, and it can be done relatively easily with two instances, but I think the suggestion from @Wilecoyote

is very good. You don’t need more than 2 to 4 scales, and everything can be done in one instance.

I’m curious to see what the end result will be. We are currently in the proof of concept phase, and I think it’s good that we are playing around with different possibilities first. What really makes sense in the end will become clear.

8 Likes

On the earlier discussion of adding extra controls, simplicity vs completeness. I think dt should expose more controls than Lightroom and shouldn’t hide complexity when to do so would obscure what’s really happening. That leads to second guessing where edits work most of the time, but when they fail, they are imposable to debug.

When things can be made more simple without damaging useful flexibility, they should be. Choice is expensive. Extra sliders take more time to learn, they need to be designed correctly, documented, the code debugged and maintained. It’s dangerous to ignore that cost.
Darktable should allow more control and complexity than other apps, but shouldn’t add low value controls for the sake of completeness. Each has to add substantial value.

1 Like

I think this thread has diverted from the main subject (Scene refered local contrast module) into a philosophical argument around control and complexity. Lets take that topic into a new thread and keep this one on track.

I would prefer not two instances since the effect of the first will impact the second one.

7 Likes

I think the module works nicely as I have said and I can quite easily grasp the main controls, however while experimenting I get to the spot that for me is similar to an issue that I have with the tone eq. and that is the choice of the “best” luminance estimator and the interaction with the other controls. For the tone eq I have landed on using the RGB sum estimator for the mask as it seems like it works best for tone eq adjustments as I tend to do them but it could be that I just found that for a few images and then held on to that rather then test several estimators for every image I edit.

So I tried an edit with this new LC module and set the boost the scale and the edge slider with the defaults. Then I wanted to experient/evaluate the guided filter mask so I started to just look at the impact of each change in the estimator…First I cycled through all 7 estimators without changing anything else and the image areas I zoomed in on would change enough to the point I couldn’t decide which I liked the best and I couldn’t find a pattern or trend. A couple for a given area would be worse but the others were basically alternate looks. Then going from there are the 4 feature extractions modes… EGIF, GF, and the average versions of both. I know from the excellent video by Boris on the tone eq roughly the difference of the first two and then the other two are a sort of hybird average of the GF’s with no preservation to reduce the effect I believe the manual says… I would likely mainly just compare the main versions of the GF and not the average ones…again the merits of ingnoring the average modes might just be a gap in my understanding of where you might really benefit from that mode.

So for now we have 4 extraction models with 7 estimators that will modify what you have set for LC with the effect sliders. I know that as with all DT modules I can choose to use only the parts of the module that I need but in this case I wish I has a little more insight or confidence how to leverage the estimators to control the output because right now for me they just produce random alternate versions that are confusing unless they go really wrong… So other than trial and error I can’t make use of them in a targeted way. If someone has insight on how to use these for certain situations or how they differ I would appreciate it.

It might be that if I went back and now changed the initial sliders for boost scale etc after changing the estimator that I could get something similar or better version between the various settings but its here if there could be some more targeted guidance so that these settings could be logically applied it might be helpful. I believe the manual just says there is no perfect one so just use the best one in the tone eq section. So again fine, but there must still be some sort of criteria that we could assign to guide using them

Where I am also heading is do we need all of that in the mask section to create a good guided filter for applying LC. Could that be simplified without losing any real control for the purposes of local contrast and then if a couple more sliders were added for finer scale or other fine tuning it wouldn’t over burden the module and it would be clear how to set it to maximize the impact of adjusting those LC sliders…

1 Like

:+1:

It’s worth remembering that this is a very new proposal, and there’s only been a couple of iterations done so far for testing. Some people are already saying, “it’s good as it is”, and while there’s some truth to that, I can’t imagine @Wilecoyote is ready to lock any features just yet.

That said, maybe there’s merit in establishing some non-negotiables as they become clear, such as:

  • works in the scene-referred part of the pipeline
  • will NOT use an equalizer UI
  • no tabs
  • no confusing luminance estimators (joking!)

I just threw these together as examples, I’m not saying they are non-negotiables. Right now, it’s still proof of concept, but it might help to keep the discussion more focused if we start to rule certain things out. Just a thought.

In my own experiments I often get similar benefit increasing the fine details as I do increasing the sharpening iterations in another module. It depends how finely you set the mask, but my concern with splitting the local contrast module into a ‘fine’ and ‘coarse’ section is that the ‘fine’ would simply be a sharpening tool. So if the module was split in two sections, I would suggest they have different defaults, but still allow the user full range of control for both - not fixed values.

I currently have a style with 4 instances of DoS - sharpening, local contrast, dehaze, and blur. They are off by default, and I don’t use all on every picture. So I am not against multiple instances of one module. If local contrast rgb was kept as is I would probably use 2 instances - to replace the local contrast and blur instances of DoS.

1 Like

I hope we see this new module in DT 5.5 in preparation of the 5.6 release. I haven’t tried the module but for me having more choices to tackle local contrast is only a good thing. Like all modules I expect it will have some advantages and some draw backs when compared to other modules doing a similar task. Then we can decide which is the best for the specific task at hand. I for one use the local contrast module a lot while I presume others defer to scene referred options of the DorS module or contrast equalizer.

2 Likes

Yes, seems like we have derailed a bit - with very interesting discussions, however :smiley:
So maybe it’s a good to summarize the findings and channel towards next steps.
First, I try to cluster the proposed designs into three clusters with pros and cons:

1. Keep initial Design

Benefits:

  • As simple to use as it’ll get
  • Does one thing well without redundancy
  • Relatively easy to implement and maintain
  • Later upgrades that add additional, independently calculated detail layers without changing the core logic can be added later with easy handling of backward compatibility

Cons:

  • Multiple instances required for multiple detail scales
  • Multiple instances are not equivalent to multiple scales within one instance because later instances process already altered image

2. Add multiple details scales relative to main scale
Masking settings refer to “main” detail scale. Only added sliders are coarser / finer scale, which use a scale that is defined in relation to main scale (e.g. 1/4)

Benefits:

  • Still relatively simple as there is only one slider per detail scale added and it can be labelled in an understandable way
  • Direct control over multiple scales

Cons:

  • Detail scales depend on main detail scale → changing main detail scale changes everything
  • fine independent adjustment of scale still requires multiple modules due to Fixed ratio between detail scales
  • Effect of Parameters like edge refinement is dependent on detail scale → lack of individual control may again require multiple instances
  • Increased computational cost

This version seems charming at first, but it feels like a half-baked solution to me because it is adding some multi-scale functionality, but then breaks with the concept of explicit control of detail scales and is hence only going half-way to a consequent multi-scale generalization of the module.

3. Add multiple independent details scales with individual control sliders per scale
Basically like this:

Benefits:

  • Direct control over multiple scales, also regarding their scale and edge refinement
  • Best substitute for multiple instances, as the scales can be controlled independently and precisely
  • Each scale’s mask can be visualized independently
  • For a user who understood the basic concept of the module, it should be intuitive that they can control different scales independently, whereas in solution 2, it may not be clear how the one refinement and feature scale slider interacts with fine, coarse etc. detail boost.

Cons:

  • Significant increase of control parameters
  • Increased computational cost

if the module should get multi-scale functionality, I believe that this one is the most suitable approach because it avoids entanglement of different parameters’ effects, is consistent with the original module logic and may still have a comprehensible UI.

4. Equalizer-Like
This would mean we have either a set of fixed scales (like tone equalizer) or freely placeable nodes (like in ART).

Benefits:

  • Direct graphical control
  • compact way to visualize settings for multiple scales

Cons:

  • First and foremost, the concept of an equalizer with a continuous domain is not applicable to this module. With the scale parameter, we pick a single scale value. There is no smooth transition between different scales happening. So the visualization with a smooth curve between the nodes would be misleading
  • A complete redesign would be required (which I would not like to do)
  • Some disadvantages of option 2 apply also here. In particular, we would also like to control edge preservation or other parameters for each scale. Also, visualization of the mask for each node is not straight forward.
  • Increased computational cost
  • No key bindings like for sliders

Whereas I like equalizer-like designs in the tone eq and color zones, I would not like it in the local contrast rgb for the previous reasons.

Next Steps
I like the list:

I still very much tend to keep the module simple with only one scale for now. This mainly has the following reasons:

  • From my experience, it is good to start small and simple with new functionality to keep it maintainable and flexible while gathering experience from users after a release. The chance is high that new learnings and ideas will come up. If the module is already complex, it will be hard to adapt and a pain to provide backward compatibility. If the module is simpler, it is much easier to adapt and develop the functionality in an evidence-driven fashion.
  • According to my understanding (please correct me if I’m wrong dear DT devs @Pascal_Obry ), a later expansion (like option 3) of the module to multiple scales will be straight forward as the scales share the same basic controls (like feature extractor) and are independent in their individual parameters. The migration path from single-scale parameters would just involve initializing defaults for the newly added scales with detail boost value 1 (=no effect).
  • To be honest, my current situation does not allow me to develop a complex module with the same effort and dedication as e.g. @kofa did with AGX. So I’d like to keep it simple and maintaineable at first!

I believe it’s relatively probable that the module will be developed towards multi-scale in the future and I am absolutely fine with that if it benefits the user and aligns well with DT’s principles.
Personally, I’d like to use multiple scales in one module in my workflow, too.
In this case, I’d prefer option 3 currently.

If I find the time, I’ll maybe also try out an implementation of option 3 to learn how this would behave. If anyone likes to do this before me, feel invited :wink:

But an initial pull request from my side would most probably be option 1 - nevertheless it is still valuable to explore options and preferences for later changes!

Luminance Estimator and Feature Extractor

Quite a lot, yes…

I guess it can be simplified.

For the feature extractors, I’d guess that the averaged ones are not needed for local contrast.
The two options would then be eigf and guided filter.
Those really behave differently, so that it might be good to keep the selector.

For the luminance estimator, I’m not 100% sure yet.
It seems to me like some situation where in 95% of cases one will do and there is no real difference between most estimators. But in some cases, you may really need it?
So maybe a little challenge :wink:
Who can find an image where different Luminance estimators make a real difference that cannot be reproduced by adjustments of other sliders?

Also here, starting simple and expanding in later versions is much easier than removing a parameter, because that is a pain in parameter migration of existing edits.
So I’d also propose to start small and maybe even remove the luminance estimator selection.

20 Likes

I haven’t tried this module yet. I am waiting for it to get merged to DT5.5. But from reading the latest post by @Wilecoyote it sounds like the simple initial design could be merged into the master and then later ‘improved’ without changing the core logic and backwards compatibility. That sounds the best approach to me as a person lurking on this thread without having tried the module yet.

3 Likes

I’ve built it so no need to wait???

2 Likes

Multiple instances should never be a con.

10 Likes

My testing is by no means extensive yet, but on a dozen or so pics, the only meaningful difference I’ve seen from the ‘euclidean’ default has been ‘max rgb’. Even then, the difference was still quite small.

for information on the luminance estimator have a look at the
curves documentation: darktable user manual - curves section “preserve colors” and for a more detailed description of the choices:
filmic rgb documentation darktable user manual - filmic rgb section “preserve chrominance”

it controls how differently colored pixels are handled as equal grey values - and that affects, if feature can be determined in the detail mask or not.

1 Like

Thanks! Exactly, I’d also explained it in brief here: Experiments with a scene-referred local contrast module - proof of concept - #42 by Wilecoyote

The important key takeaway for the local contrast rgb module:
The predictions of most luminance estimators differ only in nuances for most but the extreme colors. Their individual properties can be extremely important for use cases that require a good representation of human lightness perception. For example, when using it for perception-based color manipulation or gamut mapping tasks.
The local contrast rgb, however, mostly extracts local differences / features from the estimate for the feature map.
In this case, small local differences between the estimators have much less impact on the result as long as features are present in the luminance mask.
This is why most estimators yield very similar results in this module.

An exception are estimators that are not some linear combination of the (transformed) components. E.g. the max RGB estimator will render some very saturated areas completely flat without features (leading to no local contrast modification there) where other estimators find detail.

[Edit: don’t know the name of my own module anymore. Maybe it’s time to give up :wink: fixed this]

4 Likes