Meanwhile, I’m mostly happy with the results that I obtain with darktable 3.4.1 for low-ISO images. This is unfortunately not yet true for very noisy images. I read a lot on the web and watched videos on the topic, but I’m still struggling to obtain results that are as good as the ones of the camera’s JPEG engine. (And actually I would hope to obtain better ones…)
Is it that I’m missing something?
As a demonstration, here is a low-light still life of a bread that I baked:
This uses denoise (profiled) wavelet auto mode set to strength 3.0. I find the denoising quite effective for chroma noise, but in my opinion there is too much luminance noise left, and what’s more, that “grain” is significantly coarser than in the ooc jpeg. Is there a way to improve on this?
And here is the result of using the “non-local means auto” mode of denoise (profiled):
Grain is OK here, but there are low-frequency colored (yellowish or purple) patches all over the place that I find rather disturbing. I played with all the non-local means non-auto sliders, but did not succeed to remove these.
Since there isn’t much chance the bread will run away, I think the best fix is at capture time. The only times I’ve been happy about putting in heroic efforts to rescue an underexposed shot is when it’s a shot of something vital that I will never have another opportunity to shoot. On the other hand, that bread looks very good…maybe the opportunity is gone! Was it tasty?
This shot was taken with a pretty wide aperture (f/2.8), high-ish ISO (6400), and long-ish shutter speed (1/25 s), and yet the shot is quite underexposed. The OOC JPEG is less visibly noisy than the dt result, but still shows significant noise. If you have a tripod, bean bag, or anything else you can use to eliminate camera movement, try lowering the ISO and using a longer exposure.
This scene, of course, is just a test. I tried to post something a bit more different than a book shelf or a wall. The bread provides fine detail and “skin tones”.
In fact, I set ISO to 6400 manually. My camera has excellent IBIS, so I could have easily shot this handheld at 1/3 s and ISO 800. I could have undimmed the lighting. And I do have a good tripod.
I do not want to post Christmas Eve photos of my kids on the internet, so I made up this scene to demonstrate very much the same issues that I do have with other noisy low-light shots.
OOC JPEGs are quite decent (IMHO), so I could simply shoot RAW+JPEG, but I do like to have a consistent style, so I’d really like to be able to process all my picture in darktable.
Suggestions on how to treat such shots in darktable without heroic efforts are most welcome.
So you use two instances of denoise (profiled): wavelets for chroma noise and non-local means for luma noise. I never tried this because the manual says it’s not necessary, but your results speak for themselves.
Also, you actually use a custom Y0 curve with the wavelets instead of simply setting the Y0 curve to zero as the “chroma only” preset does. I have to examine this.
Finally, I see that you use a parametric mask to blend the result of non-local means. As far as I can tell that parametric mask is equivalent to a uniform mask, or am I mistaken? How do you choose between varying the opacity of the blending as compared to varying the strength of the denoising?
Let me describe what I am seeing. The difference between the denoising efforts so far and the JPG is that we aren’t dealing with the splotching. When we remove the fine noise, we are left with the blobs that come from noise and not from actual detail or structures. Also, the removal of the fine noise in the JPG is high but selective; fine specks and other grain remain but definitely not as super dense and sharp as the edits that left the fine noise mostly intact.
The blotchiness you saw in the non-local means is something I’m looking to deal with in Filmulator by using non-local means only for luminance, and using RawTherapee’s chroma noise reduction which works better at eliminating blotchiness.
One other observation is that this image has a lot of black spots on the knife blade after our experimental nlmeans, but they’re there even on the camera JPEG.
Yeah, there just isn’t an algorithm to compensate for ISO. There are physics involved. After the photon hits the sensor it’s just ones and zeros. Lower ISO gives the sensor more photons, that’s delicious!
High ISO;
You will alway be choosing a compromise so just tweak the parameters to suit your taste. The only way to get accurate colors in all those pixels is to give the sensor enough time to accurately sample them.
For awhile I just used non local means auto, but the chroma removal is so good with the wavelets preset that I started using it again. Then I adjust NLM to remove the luma noise completely. After that I adjust the “mix” between the two using the opacity. If there is too much NLM, then things start to look “plastic”, so I leave some luma noise in to provide “texture”.
I shoot a lot of high school sports so the action is fast and the lighting isn’t very good. I’m usually shooting at ISO 3200 or higher, up to 16000. I just took several images and used the chroma only preset, then played with the curve trying to determine what “band” the noise was in and adjusting it until I got a result I liked (most detail and least noise).
The parametric mask should be set to apply more denoising to the shadows and less to the highlights, but I just looked at it and it’s not set that way. You could just use plain opacity to vary the mix.
Since I shoot lots of high ISO images I have several presets to deal with them. I find that demosiacing makes a difference, and VNG4 seems to work best for me with my cameras (Canon 7D & 7DmkII). I also use the color smoothing, usually set to 3. Beyond 3 I don’t see any difference. When the ISO’s get really high I add in hot pixel removal. I usually don’t set denoise with a preset because of the processing required. I leave it to the end and just apply it with a style.
I think it looks fine with the built-in ‘wavelets: chroma only’ preset. It’s a bit grainy, but I prefer luma grain over oversmoothing. No need to pixel-peep; you don’t do that with Christmas Eve photos either, right?
I agree that with wavelets only it looks OK when not pixel peeping ;-).
But I can’t help to be a perfectionist and I really like the combined method of @wpferguson. To my eyes it does not destroy any details, but the noise is rendered much less offensive. Here is a direct (pixel peeping) comparison. The noise treatment is the only difference: Bill’s method on the left, “chroma only” preset on the right.
I played with demosaic parameters both with denoise off and in wavelet mode. To my eyes there are subtle differences, but no unambigious improvement.
Yes, this pinkish color cast remains after wavelet denoise. It’s not present in the camera JPEG. Could you give me a hint on how to best remove it with darktable?
Rawfiner does a 2.6 video but he dives in to all the denoise method s over a range of Iso settings. I think he used bilateral denoise now renamed surface blur to deal with them channel by channel on r GB channels…it’s a great foundational video if you have not viewed it
demosaic by design makes only marginal changes but they are always good to make if you are into pixel peeping.
There is another approach to noise reduction — contrast equalizer module with quite good module presets. It allows good separation between high and low freq. noise. This is how I removed that pinkinsh color cast.
Rawtherapee does better job in noise removal in it’s default settings. Meanwhile dt requires some background understanding.
@alpinist sensor cooling and low ISO are the best ways of noise reduction Still wonder why they don’t use sensor cooling in consumer grade cameras like astronomers do…
Don’t expect a lot from software. This camera made very noisy picture at ISO 6400 with some totally black pixels (dead?) and big pinkish artifacts. There is little real information behind that random mess.
Even more, adding little noise can improve perceived image quality (especially printed) just like they add noise in lossy sound codecs to hide artifacts.
P.S. I don’t like builtin JPEG result. Those black pixels are very visible, no matter what picture scaling algorithm you use to see final picture on your display.
the green spots on the knife are, in my opinion, rather reflections from the bread.
i also don’t see the area you are focusing on in the image as meaningful, since it is the blur area in the image. this is visually always associated with “more noise” due to the clearly less recognizable strctures (even in low iso range)
i also think that a combination of non-local and wavelet denoising will do an extremely good job here.
Still wonder why they don’t use sensor cooling in consumer grade cameras like astronomers do…
