Changes in noise reduction for darktable 2.7/3.0

Likely you may run into a kind of saturation losses, when you see the pic with 0% zoom. Be careful!

Uhm, let me understand then. What is the best demosaicing? For Fuji cameras, in my experience the Markensteijn 1-pass is very often “good enough”. The color smoothing is something I’ve never understood so much, so should one turn it on only for weird artifacts like the one I’ve shown above, or would it always “destroy” a little bit the minute details and colors?

Daer @rawfiner,

I appreciate all your efforts, denoise profile really made a big step…

What I think was probably a little regression, was the changes in the pre-set for croma-1st-instance and luma-2nd-instance.

In the past it was like this:
DNP_OLD_Screenshot_20191105_214557

and now it sems to be like this:
DNP_NEW_Screenshot_20191105_214742

Which finally in my opinion (if one still goes that path) leads to some regression, as I found this:

  • on the right is the “new”
  • I did have some examples, where that approach even created more grain, than without any denoise :blush:

I’m not a pro on this too. I learned it once from Harry Durgin (for milkyway shots) and later figured, that it also costs some colours. So just be careful and use snapshot to find the right trade off.

2 Likes

I can’t speak about dt, but in rt Markensteijn 1-pass is good for high-ISO shots, while Markensteijn 3-pass is good for low-ISO shots, while Markensteijn 1-pass + fast and Markensteijn 3-pass fast may be preferable to the ones I mentioned above.

Ingo

1 Like

I am no more using the blending modes. At this stage of the pipeline, it is preferable not to use color-related or lightness-related blending modes. Allow me to quote @anon41087856, as he explains it very well in this other thread:

So now, I am always using only one instance of denoise profile, usually in non local means mode, and I use the central pixel weight parameter to control the amount of reduction of “luma” noise vs “chroma” noise.
About the adjust autoset parameters slider, this slider maybe won’t give you big differences in wavelets mode where it changes very few parameters, but it gets more useful in NLM mode as it allows to set quickly almost all NLM parameters at once.
Basically, increase this slider’s value if the image is underexposed and you compensated the underexposure in post processing.

Increasing the slider’s value will:

  • increase patch size
  • increase scattering
  • reduce shadows preservation
  • reduce bias correction

Indeed, NLM was only good for low ISO images before, but now it is working well even for high ISO images, and now I almost always use NLM whatever image I have to denoise. :slight_smile:
The change that made it behave better for high ISO images is not the central pixel weight though, but the scattering parameter.

5 Likes

@AxelG
The only change made to the preset was to switch from color and lightness blending modes to HSV color and HSV lightness to avoid highlight clipping (and add the 2 new scales in wavelets).
Both presets were already using wavelets in dt 2.6 :wink:
I don’t know where your NLM based preset comes from. See for instance the 2 presets in dt 2.6 here at 16:06: https://youtu.be/NFhkdzFFeEw

Anyway, the presets were left but my advice is to stop using them, and to use only one instance instead, without color or lightness related blending mode.

2 Likes

Oh wow, that’s a radical change then!

Thanks for the info, I think this has to be specified loud and clear and written in the manual and perhaps add a preset too.

This will make denoising also easier for beginners I believe, no more fuffing around with blending modes and creating two instances etc…

Also, I have never quite been able to grasp the immediate consequences of a lot of Aurelien’s fights with some of Darktable’s tools and decisions, esp.regarding lab vs rgb, right now I do understand a little bit more. I would probably need to put up a post asking for some more direct information, such as: am I “allowed” to do tone curves in lab mode? or is preferrable to do tone curves in rgb mode selecting one of the options to “preserve colors” (but then which one? Max rgb, average, etc…).

In the meantime, thanks so much for explaning how the new denoise works and how it’s meant to be used.

1 Like

I had no idea about these differences… I have quickly checked again dartkable’s manual and this is the section on demosacing:

The default algorithm for X-Trans sensors is Markesteijn 1-pass, which produces fairly good results. For a bit better quality (at the cost of much slower processing), choose Markesteijn 3-pass. Though VNG demosaic is faster than Markesteijn 1-pass on certain computers, it is more prone to demosaicing artifacts.

So obviously I understood Markesteijn 3-pass >> better than 1-pass, end of story!

If the presets do no reflect the correct way to use the module any longer, shouldn’t they be removed?

2 Likes

Dear @rawfiner
I see your point, I checked the video, I cannot explain as well. Anyhow, I agree with you, don’t get too much headaches and use the new NLM (auto if wanted)

I donno whether you have tested with astro fotografie as well. IT IS AMAZING!!!
Thank you for everything!

Sincerely
Axel

3 Likes

I’m the one asking to keep them :slight_smile: I do still use the chroma one, it produces grains on the picture but the print is really clean anyway. I’ve just printed a set of 40 pictures in A3+ and A2 for an exhibition, all pictures shot between 3200 to 5000 ISO and the print are clean. The visible grain on screen when you zoom is un-noticable on paper and I find the render better and sharper.

That being said, if it proves to raise more confusion I’m ready for a PR to remove them as I can easily add my own presets anyway.

2 Likes

Maybe just rename them then? People won’t read the release notes and will still use the old way :slight_smile:

1 Like

Alright, tried the new denoising technique on a few images and I remain unconvinced.

Not the new parameters or the recommendation/suggestion to use NLM in auto mode “always”, i.e. regardless of whether it’s chroma or luma noise.

I’m not sold on the idea to avoid blending modes. You specifically said that you are not using blending altogether, but more generally that no one should use HSV color or HSV lightness blending modes.

First example (this particular image is probably worst case scenario, a daily scene shot at ISO 51200 with my Fuji):

On the left, NLM auto, all defaults. On the right, the classical preset for chroma noise (wavelets mode blended 100% in HSV color). Screenshot from 100% zoomed in image.

The NLM auto applied brutally like so gives me the impression of the usual heavy denoising applied in camera which I dislike, with mushiness and destruction of details (I know, mostly noise but the impression that I get is of higher definition and more realistic == less “digital”).

So now compare the classic denoising as above (right) with the NLM auto blended in normal mode at 50% (left):

This is better than before for me, but is it better than blending in hsv color?

Same photo, a detail of the sky:

(Zoom all the way in to see what I’m talking about).

On the left again NLM auto 50% blend normal; on the right wavelets blend 100% HSV color. What’s better? The trees are possibly more detailed on the right, and the sky – yes there’ a noisy pattern but it’s mostly monochromatic and more “organic”, I think I like it more!

However, I move to another image, this is shot with a m4/3 camera at very high ISO (16000!) under artificial light. Again just an example! Apologies for the very poor snapshots I’m using here!

In this case the comparison is between NLM-auto-no blend (below), with rather aggressive parameters (central pixel weight 1.00, adjust autoset 3.59, strength 1.3) and the standard chroma denoising above (default parameters as in previous examples). Now I’m confused because the first method is much better! There still is some ugly chroma noise left but that is impossible to remove I think.

So there you go. I think I had a point,. that blending in HSV color still works better but I have a random collections of (mostly poor) images that maybe are better processed with the new approach, i.e. without any blending.

Perhaps for the rest of today I should process clean, low ISO photos.

:thinking: Better seems to do with the fact that one side of the comparison has more grain. If we matched the graininess, would your choice still be the better one?

I haven’t gone through thorough tests as @aadm, but have picked some late edits and swapped the old couple denoise chroma + denoise luma for the new nlm and got excellent results, after I learned in this thread how to do it right.
In the end, I find the new nlm easier to tweak, and in a more flexible way. For example, removing (or mitigating) dark blotches in very underexposed or high iso images, which, in the old way, I had enormous difficulty to deal with.
All in all, I’m very happy with it so far.

I am also quite happy with the current denoise.

One thing very interesting, when I work in 100% mode, the picture looks sometimes a bit worse or at least not that pleasing, but when I zoom out to full view (finally relevant), suddenly you can see a big improvement in microcontrasts with the new NLM method

I just sometimes would like to have a wider range of the auto-sliders. Sometimes I have them at the end and still not totally happy :slight_smile:

In my humble opinion, lowering opacity works well for low and medium ISO images, but less well in high ISO images.
In that case if you want grain, I would push central pixel weight, and reduce strength if needed.
I almost always use NLM, but I also always tweak its parameters, I find the default force value almost always too strong :wink:

Try doing the same approach (tweaking the 3 parameters instead of just reducing opacity) with the others images :wink:
Also, it’s not a big deal if some chroma noise remains at 100% zoom level, if you don’t see it when zoomed out :slight_smile:

Anyway, things I say are only advices, you can still use color blending modes if that suits your need. :wink:

Happy to hear that :slight_smile:
Thanks!

You do have a wider range, for almost all parameters of denoise profiled :wink:
You can enter values outside visible bounds by keyboard. The real ranges are:

  • [0;12] for patch size
  • [1;30] for search radius
  • [0;20] for scattering (though the value 1 is probably enough even for very high ISO images, and you will probably destroy local contrast by pushing the value too far)
  • [0;10] for central pixel weight (though very large value makes the result very similar to having 0 in patch size)
  • [0;1000] for strength
  • [0;1.8] for preserve shadows
  • [-1000;1000] for bias correction
  • [0;1000] for adjust autoset parameters
1 Like