The darktable magicians have crafted something again - detail mask

[s7habo (Boris Hajdukovic)]

Based on @heckflosse idea, @hannoschwalm has implemented a new masking option that is hard to beat!

Soon in new darktable version 3.6 we will have an additional option to refine the mask by targeting the areas with higher or lower detail density and strength.

When this new slider appeared today and I started playing with it, I almost fell out of my chair!

There are now countless uses that are now open with this option.

I will not go into detail here, but after an hour of playing, just touch on the surface, what I came up with what you can achieve with it.

Strengthen depth of field.

Lowering coarse part of local contrast on low contrast area with contrast equalizer.

Masked areas with low details:

before:

after:

Sharpen

Boost local contrast only on the already detailed areas with high frequency details. Also with contrast equalizer.

Masked only high frequency details:

before:

after:

This is an excellent way to sharpen only what should actually be sharpened.

Local contrast boost on the areas that look very flat. These are often house walls, snow, sky, etc.

Masked areas with very low details:

before:

after:

Reverse to sharpening, this way you avoid increasing the local contrast in the areas that already have a strong local contrast.

Soft focus

Especially in landscape shots and portraits, one often struggles with too strong, rough textures in the photo.

With the detail mask you now have a nice way to weaken the areas with strong textures a bit by lowering the local contrast in these areas.

The areas with strong texture are masked here:

before:

after:

Here is another example with soft focus:

I can think of a lot of other applications (portraits etc.) but to raise curiosity, this should be enough for now.

In any case, great work @hannoschwalm, @heckflosse and all the others involved!

Thanks a lot for that!

[elstoc (Chris E)]

Thanks for this @s7habo. I haven’t really played with the detail mask very much and was struggling to think of use cases for it. This helps a lot.

[Dave22152 (Dave Goldberg)]

That’s just fantastic. This should really come in handy during denoising, where I want to target low frequency detail like sky and water and leave high frequency detail alone.

[anon41087856]

@Dave22152

This should really come in handy during denoising, where I want to target low frequency detail like sky and water and leave high frequency detail alone.

Noise is high-frequency, so details and noise will be masked altogether (though there are some mitigations strategies to somewhat reduce noise sensitivity, but noise is still read as high-freq nonetheless).

Which is actually the whole conundrum of denoising. You bet if it was as easy as masking out legitimate details, we would be feasting on DxO Prime ashes already.

[PhotoPhysicsGuy (Bob)]

Noise is a broadband random signal. Just like any ground truth feature is a broadband signal. The conundrum of denoising is that it is hard for an algorithm to separate random noise structure from actual (sometimes random) image content, for an algorithm they can look similar.

[Dave22152 (Dave Goldberg)]

Thanks for the clarification, and of course, what you say makes sense. A lot of times when I denoise, I’ll manually mask around the subject to preserve details where noise isn’t as apparent.

The photo of the mask around the heron was a perfect example. Still, I appreciate this new development and I’m sure it’s going to help in a lot of ways

You guys are really outdoing yourselves these days.

[anon41087856]

@PhotoPhysicsGuy

Noise is a broadband random signal. Just like any ground truth feature is a broadband signal. The conundrum of denoising is that it is hard for an algorithm to separate random noise structure from actual (sometimes random) image content, for an algorithm they can look similar.

I don’t understand what you mean by broadband. It happens at sample scale, is has the dimension of a pixel, it gets removed by a low-pass filter, it’s high frequency to me. I’m talking about spatial frequency here.

[PhotoPhysicsGuy (Bob)]

@anon41087856

I don’t understand what you mean by broadband

The spectral power density of noise is not just high-frequency. In case of white noise, it’s constant over the whole spatial frequency spectrum. Pink noise for example has a power distribution that goes with \frac{1}{f}. This is what I mean with broadband.

There are many ‘colors of noise’ https://en.wikipedia.org/wiki/Colors_of_noise#Purple_noise.

A low pass filter (maybe a box blur or a gaussian blur) is a very very crude implementation of a denoiser.

[Peter]

I like the results from this one. Have been playing around with it for the last couple of days.

Exporting/importing photos in full 32-bit representation (to train a neural network for image denoising)

Hello all, a while ago I made a dataset of clean/noisy images to train a neural network for image denoising, published under Natural Image Noise Dataset - Wikimedia Commons (there is also a paper associated, http://openaccess.thecvf.com/content_CVPRW_2019/papers/NTIRE/Brummer_Natural_Image_Noise_Dataset_CVPRW_2019_paper.pdf ) The images were processed in darktable with all the steps applied except for sharpening (which amplifies noise), and aligned with Hugin tools, and the result was rather e…

[MarcoNex]

@s7habo Many thanks for this.
I tested the detail mask before the merge and immediately realized it’s a super cool feature.
I was looking for use cases and indeed you provide convincing ones. I really believe you should contribute to the official DT documentation with real use cases before/after, taken from your fantastic edits. One image is worth a thousand words!

Coming to first case you describe (artificial bokeh), wouldn’t the low pass module be more appropriate than contrast equalizer? The former effectively blurs areas similar to an out of focus lens, the latter only reduce the contrast.

[elstoc (Chris E)]

@MarcoNex

I really believe you should contribute to the official DT documentation with real use cases before/after, taken from your fantastic edits. One image is worth a thousand words!

We try to avoid very detailed “how to” guides with lots of images in the user manual, but I absolutely want to include some brief use cases in the documentation. Just a few sentences though. As @s7habo has illustrated, there are a number of useful ways to use this module but including them all with before/after would be too much for the manual.

We have other places for step-by-step guides. For example, the darktable blog.

[Soupy (Tim)]

I am very excited for this mask, particularly if darktable implements capture sharpening, which I believe is on the way, though not sure which release is targeted.

In regards to contrast equaliser, isn’t it a wavelets module, thus already split into high/low frequency? Ie. If we move the right most point up to sharpen only high frequency detail, then slide the detail mask to effect only high frequency detail, wouldn’t the detail mask be doing very little? Using your bird image as an example. In the first shot you have contrast equaliser 2 turned off. While in the second, you have it turned on. A more apt comparison would be first shot - contrast equaliser turned on with setting to reduce low freq detail. Second shot - contrast eq as per first shot, but with details mask also used). Same goes for flower image and forest image.

[s7habo (Boris Hajdukovic)]

@MarcoNex

Coming to first case you describe (artificial bokeh), wouldn’t the low pass module be more appropriate than contrast equalizer?

This is theoretically more correct, but you will increase halo around the sharp subject:

You can do this in GIMP by cutting out the subject, copying it to the layer above the original and replacing the subject with the surrounding background in the original.

By the way, this is also a nice usage - an interesting way to get the Orton effect:

before:

after:

with harmonic mean blend mode:

@Soupy

In regards to contrast equaliser, isn’t it a wavelets module, thus already split into high/low frequency?

Yes, but it does it over the whole photo. In relation to the photo with heron, if I reduce the coarse part of the local contrast, heron will also be affected, i.e. it will look sharp but very flat because it will also lose the coarse part of the local contrast like the background:

The detail mask - especially in combination with a small amount of feathering - ensures that the sharp areas are not affected at all, i.e. it keeps its sharpness and coarse local contrast.

The same logic applies to the rest of the photos from the demonstration above.

@elstoc

We have other places for step-by-step guides. For example, the darktable blog.

I am ready to provide the content and material for blog entries, if someone would be willing to accompany me technically and correct the content and language.

[pass712]

Thanks @s7habo for posting / explaining and thanks to the dev-team ! Didn’t notice this nice little feature.

[elstoc (Chris E)]

@s7habo

I am ready to provide the content and material for blog entries, if someone would be willing to accompany me technically and correct the content and language.

I’m sure @paperdigits can help with the technical stuff and I’m more than happy to help with copy-editing (and technical stuff if I can). As a starter, you could take a look at the blog post that was made for the last darktable release, here.

Also, we’ll be doing a new blog post announcing darktable 3.6, and some brief examples would be useful there as well.

[anon41087856]

@PhotoPhysicsGuy

The spectral power density of noise is not just high-frequency. In case of white noise, it’s constant over the whole spatial frequency spectrum. Pink noise for example has a power distribution that goes with \frac{1}{f} 1f\frac{1}{f} . This is what I mean with broadband.

That’s noise over a temporal wave. Photosites bin photons and integrate the light wave magnitude over time, we don’t have Hz and we certainly are not in Fourier space. Any image denoising algo off the shelf is a low-pass filter, no matter how clever it is.

[lockdown]

Pure genius. Thanks to those who implemented it.

[Aliks]

Advancing in leaps and bounds. With plenty of time on my hands due to lockdown, I have gone over my 2020 photo collections twice already. Each pass has dramatically improved many photos, using proper scene-referred workflow, then using color calibration properly, then using the beta version of colorbalancergb.
Now I’ll have to go back and use the parametric mask with the details threshold slide.

Where will it all end???

[Claes]

@Aliks

Where will it all end???

Hopefully, it will not end

[PhotoPhysicsGuy (Bob)]

I don’t know where to start since this is information theory 101.

@anon41087856

Photosites bin photons and integrate the light wave magnitude over time…

Not what I meant. Take matlab or python or whatever, generate a 2D ‘pixel’ grid and fill it with your favourite noise type. Then do a FFT, look at the spectrum of spatial frequencies. Does that plot contain just high frequencies? Remember: a fourier transform is ‘just’ switching the basis of how you represent your measurements. So if you want to, yes you certainly are in fourier space. That’s exactly why I chimed in, as I saw a…let’s say bold statement…regarding spatial frequencies (you changed the basis set in order to make that statement anyway) of noise.

@anon41087856

That’s noise over a temporal wave.

Well. Noise is a random distortion of whatever you want to measure, however you sample it. Spatial frequencies may not be normalized to the unit of seconds but to whatever image dimension you want…picture height, block-size…but the same priciples apply. Take a photo of a flat surface at low light and do a FFT of the resulting image, you’ll see that the noise spectrum contains all frequencies at various intensity levels.

@anon41087856

Any image denoising algo off the shelf is a low-pass filter, no matter how clever it is.

No. Take for example DCT-denoising or Wavelet denoising. They do coefficient shrinkage either in discrete cosine frequency domain, or in wavelet domain (which is a more complex change of basis set than a fourier transform, it’s multiscale by nature o the transform). nlmeans has problems with low frequency noise and that’s one of the reasons why multi-scale denoising schemes are investigated. Multiscale in order to attack not just high frequencies for algorithms that out of the box just chuck away high frequency detail.

The detail mask dicussed in here, is one of many tools in denoising to restrict effects to ‘flat’ areas. In terms of frequency, those are low-frequency. Leaving edges alone is very important because the fourier transform of an edge is…? You guessed right, broadband in spatial frequency terms. You have broadband noise, and broadband features and you need algorithms to distinguish random noise from real structure.

Of course the detail mask is good for many other things as well like we see in this thread (feature selective sharpening and local contrast adjustments).

Cheers.