Possible to achieve good basic chroma noise reduction with a preset?

[alpinist]

This is a continuation from a different thread.

@priort

@alpinist

Now if “someone” one day would find the time and inspiration to improve Darktable’s noise reduction, this software would become almost perfect from my point of view.

Post some examples of where it fails. I have ON1 photo that touts its fancy nonoise AI and usually the results with DT are as good if not better. Maybe you are simply using profiled denoise in DT and the profile for your hardware maybe could be better???

I suspect you have watched @rawfiner videos so as to get the most out of the 4 or 5 ways to denoise in DT??

In any case an example of “poor” performance would be useful on a path to better…

Yes, I watched all the videos, I spent hours playing with the possibilities, and I already had started a discussion on the topic some time ago.

There, people were friendly to suggest various ways of reducing noise through combination of modules, and their results are quite satisfactory, but using them requires tweaking on an image-to-image basis. This reminds me of the situation with highlight reconstruction before dt 4.2, and that’s why I made my above remark.

Here is an example of what I’d like to achieve. The following is a very small crop (covered by fair use, I assume) from a low light sports shot by dpreview. They developed it with Adobe software that I do not have access to. I don’t even like their edit (I find it way over the top), but I like Adobe’s denoise. Luminance noise is well visible but is not disturbing. It even adds a certain quality to the image:

Here is what I obtain using the denoise (profiled) preset “wavelets (chroma only)” and otherwise trying to more or less replicate the dpreview edit (without quite as much contrast and saturation):

basketball-0.orf.xmp (8.0 KB)

My edit attempt with darktable has a blue-grayish haze that I find difficult to get rid of reliably. The face is not so bad, but the background is full of dots that remain very visible even if exposure is tuned down to make the background darker. Increasing the strength of noise removal does not really help either.

Ideally, such basic noise removal could be achieved by simply activating a module with its defaults or with a basic preset that could be activated by default for all images, fire-and-forget.

The noise in the above image is mostly photon noise, and this is a basic physical phenomenon that does not depend on camera/sensor details. Apparently, algorithms do exist to contain it that provide better results out of the box than what can be easily achieved with darktable. I would love to have the time to work on this topic myself, perhaps one day…

Until then, I’m happy to hear any suggestions how to deal with high-noise images without spending hours on hand-crafting and tuning presets.

[apostel338 (HP)]

I think the profiled NR of DT does great on chroma noise, without the need to messing around with it more than necessary.

But there is a serious problem with your chosen sample:

There is no noise profile for this camera. If you simply chose the photo independently of the camera as an example, my personal subjective advice would be: save your time and choose an other one. If you really own the E-M1X, I’m afraid you currently have bad luck. (There are certainly some tricks for this too, but fortunately I haven’t had to use them so far.)

If you want to help helping:

How to create camera noise profiles for darktable

[alpinist]

I use an OM-D E-M5 Mark III that has a very similar (or the same) sensor, but for which darktable does have noise profile. However, there are no comparable images (low light with ACR edits) on dpreview for my camera. I myself do have low-light images, but I do not have access to non-free raw development software.

So here’s a test image of mine (I hereby put it into the public domain) that shows the haze/dots problem even clearer. It’s a basic edit where denoise (profiled) is enabled with the built-in “wavelets (chroma only)” preset. As said, a noise profile for this camera does exist within darktable.

210210_211035.orf (20.3 MB)
210210_211035.orf.xmp (6.3 KB)

[paperdigits (Mica)]

@apostel338

If you really own the E-M1X, I’m afraid you currently have bad luck

Not really, the nosie profiles are pretty easy to shoot and generate, and you can load them without waiting for dt to add them.

[apostel338 (HP)]

Chroma only preset, indeed not that good here:
(I also don’t have own photos from my Nikon where this preset does a reasonable job, although raising the shadow slider improves )

••••••••••••••••••••••••

Module default with wavelet (auto):
(This is my personal default setting which I hardly ever touch)

••••••••••••••••••••••••

Tweak a bit for more focus on chroma:

For me, that’s a good job on chroma.

[alpinist]

Thank you @apostel338! This looks quite good. However, I have trouble to exactly reproduce your edits. Would you mind sharing your XMP files?

[alpinist]

Oh, I think I understand: you also adjusted the graph.

[apostel338 (HP)]

Yes, this is the stuff I set up:

If you want more control, these wavelet manual settings seem to work well:

There seems to be a very little purple color cast in the darks under the sofa that you can eliminate with the manual settings. You will need to adjust these settings individually for each ISO sensitivity.

I use dev/nightly builds, so if you have 4.0.1 stable, the xmp might not open correctly for you.

210210_211035.orf.xmp (10.0 KB)

[rawfiner]

Hi!
For this camera darktable does not know the profile, so you should adjust the strength. I lowered it to 0.5.
Also, unfortunately sometimes for indoor shoots, the modern white balance makes chroma denoising less good. This is due to the fact that Y0U0V0 conversion is not completely white-balance agnostic (the Y0 coefficients can be white balance agnostic depending on the checkbox option, but the U0V0 coefficients are not).
Here I am using the color calibration to do the white balance (result is similar as yours):

And here only the white balance module (noise in the background is clearly less visible):

Hope it helps

[alpinist]

@rawfiner

Also, unfortunately sometimes for indoor shoots, the modern white balance makes chroma denoising less good. This is due to the fact that Y0U0V0 conversion is not completely white-balance agnostic (the U0V0 coefficients can be white balance agnostic depending on the checkbox option, but the Y0 coefficients are not).

Thanks, that’s very interesting… Do you understand why?

(I don’t even understand why the modern white balance is even necessary. White balance corrects a scene that is not illuminated by white but by some other color (R,G,B). To correct, it seems sufficient to divide all reds by R, all greens by G and all blues by B. This amounts to individual linear scaling of the color channels (or, equivalently, the transformation of the color triples by a diagonal matrix). The modern WB instead is somehow a generic matrix transformation.)

[rawfiner]

@alpinist

a scene that is not illuminated by white but by some other color (R,G,B).

I am not a white balance expert, but the scene is illuminated by a light that has a spectrum: R, G, and B coefficients are not a precise representation of that spectrum.

@alpinist

Do you understand why?

The Y0 coefficient (~luma) is computed as R+G+B, with white balance removed (or not depending on the checkbox). The aim is to minimise the noise variance as R, G, and B have (mostly) independent noise.
The formula for U0 and V0 are U0 = R-B and V0 = R-2G+B.
For U0 and V0, the aim is to cancel out the signal, to keep only noise (mostly chroma noise).
When we have a bad white balance before denoise profiled, this cancellation probably does not work well, because R and B can have different magnitude.

[alpinist]

@rawfiner

@alpinist

a scene that is not illuminated by white but by some other color (R,G,B).

I am not a white balance expert, but the scene is illuminated by a light that has a spectrum: R, G, and B coefficients are not a precise representation of that spectrum.

Seems to be related to brain+eyes: color calibration - Brainblasting... - #16 by anon41087856

@rawfiner

@alpinist

Do you understand why?

The Y0 coefficient (~luma) is computed as R+G+B, with white balance removed (or not depending on the checkbox). The aim is to minimise the noise variance as R, G, and B have (mostly) independent noise.
The formula for U0 and V0 are U0 = R-B and V0 = R-2G+B.
For U0 and V0, the aim is to cancel out the signal, to keep only noise (mostly chroma noise).
When we have a bad white balance before denoise profiled, this cancellation probably does not work well, because R and B can have different magnitude.

Thanks. I tried moving the denoise module after color calibration, but this doesn’t help.

[Terry (Terry Pinfold)]

@alpinist

but using them requires tweaking on an image-to-image basis

Since the latest stable release of DT in the midyear I have found that denoise (profile) module does an excellent job of both chroma and luma noise at default values for most of my shots. If I have crazy amount of noise such as using my compact camera at 12800 ISO then I need to delve deeper into using multiple modules. This usually includes using RAW denoise + Contrast Equalizer module and increasing the strength of the denoise (profile) module to 1.5. I find it an untrue statement in my opinion that DT requires tweaking noise on an image to image basis. I have a preset I apply to my images at the start of processing which always includes denoise (profile) module at the default settings. For most images it works wonders. In the 2021 releases of DT I only used the the built-in “wavelets (chroma only)” preset and tackled luma noise separately, but now with the latest versions I great results straight out of the box with the denoise profiled default values.

[PhotoPhysicsGuy (Bob)]

@alpinist

I don’t even understand why the modern white balance is even necessary.

Keywords to look up: Discounting the Illuminant, von Kries assumption, von Kries-Ives model, Bradford transform, Color Adaptation Transform, Gamut, working Gamut, handling out-of-gamut values, CAT16, non-linear CAT models,…

@alpinist

Seems to be related to brain+eyes

Everything color/image related is brain and eyes. Color is a percept. Blackness, Lightness are percepts. Every optical illusion you know speaks for the complexity of this psychovisual system. Even achromatic images (black and white) have a ton of perceptual effects at play.

[alpinist]

@Terry

Since the latest stable release of DT in the midyear I have found that denoise (profile) module does an excellent job of both chroma and luma noise at default values for most of my shots.

Could you try the above shot with the rocking horse and let me know what you think? I don’t think that any defaults work well on it (I’m using the current development version of darktable).

This is from a camera with crop factor 2 at ISO 6400, so the noise is not yet “crazy” in my book.

[123sg (Steven)]

@alpinist

I don’t even understand why the modern white balance is even necessary

I don’t doubt that it is useful for some things, but for me personally I find it unnecessary - I’ve quite recently reverted to the ‘legacy’ wb settings in preferences. I like the color calibration module for other stuff, though, the various channel mixer bits and so on.

[alpinist]

[quote=“rawfiner, post:9, topic:33757, full:true”]
Also, unfortunately sometimes for indoor shoots, the modern white balance makes chroma denoising less good. [/quote]

Indeed, even the nice result of @apostel338 (Possible to achieve good basic chroma noise reduction with a preset? - #8 by apostel338) improves a lot when modern WB is switched off. The residual blue haze under the sofa disappears.

The effect seems stronger for shots that are illuminated by light that differs a lot from standard white (e.g. very low color temperature).

So, unfortunately, it seems that darktable’s denoise and its modern white balance are not 100% compatible. Did @anon41087856 ever comment on this?

[Terry (Terry Pinfold)]

To be honest the noise did not worry me in this shot due to the nature of the subject. Film used to have grain so I am tolerant to small amounts of noise. In my processing I have applied denoise profile at the default value and strength. I then applied local contrast module at default value value which reveals noise further. I then applied astrophoto denoise at default values. I am reasonably happy with the result. Here is a screenshot of the result.

EDIT: I just took a look at the denoise presets in diffuse or sharpen module and they looked really good at further reducing noise with out softening image noticeably. DT really has some great denoise choices to explore. Also using parametric masks more denoiing can be applied to the shadows where it is most needed and less to the bright subject of the girl. I just didn’t invest the time to do this on this image.

[priort (Todd Prior)]

For this I use surface blur which is really the old bilateral denoise… add that to profiled denoise… Here I have not even tweaked the radius or channels. I suppose you could tweak the luma curve in denoise profiled for this … I don’t bother. I will also use astro denoise with a max patch size and drop the strength… it can be very good at grain removal and doesn’t kill detail if used correctly…

PC170315.ORF.xmp (13.3 KB)

[priort (Todd Prior)]

210210_211035.orf.xmp (18.5 KB)