Had a bit of completely unscientific play with noise reduction, a rather naive implementation, I’m afraid. But it was fun.
No colour management (no camera matrix applied). White balance multipliers taken from darktable, but the image was extracted using dcraw -D -4 -T, and the resulting TIFF was processed. Applied a simple 1/2.2 gamma curve to the LX7 image to get something viewable; the Nikon image was very dark, I used 1/4.2 to display it.
The original and processed images were not equally bright, so I tried to quickly bring them closer in Gimp using the Levels tool.
The raw panes of the original images were converted in 16x16 blocks using 2D FFT. Many random samples of that size were taken; the one with the lowest sum of the complex magnitudes was taken for each pane as the ‘noise reference’. Then the whole image was processed in 16x16 blocks, the magnitude of the spectral value of the noise was simply subtracted from the spectral magnitudes of the blocks on a frequency-to-frequency basis, the ratio of the remainder and the original magnitude was used to scale the frequency component of the block, then the inverse transformation was performed. I guess it makes those actually knowing something about signal processing cringe.
Demosaic was done by simply extrapolating the missing colour components by averaging the closest neighbours of the corresponding colour.
High-ISO Nikon D7000 shot - here, the trees and the shrubs in the background became greener:
Thanks – but there are many topics about noise redution. I’d like to use this one to gather feedback on my approach, what I did right, what I did wrong, what would be interesting to try. Not to get to the pro level, just to satisfy my curiosity.
I can’t say that I understood what you have done. For me its looks like you have done a very good job removing the color casts of the noise but not such much the noise itself.
I didn’t target colour casts – in fact, I did not target anything.
I took the image, tried taking many 16x16 samples from random locations to find ‘smooth’ locations for all 4 Bayer ‘panes’ (R, G1, G2, B). I hoped to find one block for each pane where there was minimal actual detail, the ‘signal’ (variations inside the block) being mostly (due to) the noise. This was done by taking a 2D-FFT of the sample (actually, creating a 32x32 square, filling it with the average of the 16x16 pixel values of the sample, and copying the 16x16 patch in the middle – if I understood correctly, that should reduce the effects of Fourier assuming all signals to be periodic), and then simply adding up the magnitude for each frequency component.
Then, process the raw panes in 16x16 blocks (2D FFT using the same padding as described above), subtract the magnitude of (what I hope would be) the noise (determined in the previous step) from the magnitude of each frequency in the sample, and inverse-FFT the remaining signal, which is then used as the ‘denoised’ values for the block.
Finally, demosaic the ‘denoised’ panes as usual. Since I was ‘taking energy out of the image’, it became a bit darker than the unprocessed version, so I boosted the processed image to have the same average value as the original.
That cast was basically the black level (my code did not employ any black level, unlike darktable and other raw processors) If I zero out the component for frequency = 0 (so, the constant offset), the cast stays. However, there is some noise reduction:
Compared to denoise (profiled) (not quite the same exposure and contrast, so not an exact comparison):
Left: darktable, right: silly filter (plus, mine was demosaiced with bilinear, which is anything but good and sharp, so it may have helped with the noise supression :-))
The denoising algorithm uses bayer RAW images as input. Motion Cam treats the RAW data as four colour channels (red, blue and two green channels). It starts by creating an optical flow map between a set of images and the reference image utilising Fast Optical Flow using Dense Inverse Search. Then, each colour channel is fused using a simplified Gaussian pyramid.
Demosaicing
Most modern cameras use a bayer filter. This means the RAW image is subsampled and consists of 25% red, 25% blue and 50% green pixels. There are more green pixels because human vision is most sensitive to green light. The output from the denoising algorithm is demosaiced and colour corrected into an sRGB image. Motion Cam uses the algorithm Color filter array demosaicking: New method and performance measures by Lu and Tan.
Thanks for the pointers. At this point, I’ll certainly not add a new demosaicer (I may upgrade to something better than bilinear, but for direct comparison with other tools, I’d probably take RCD). I’ll have to read on the noise reduction method you pointed me to, currently it’s all Greek to me (but I’m here to learn, so that is a good thing).
I get ringing, and this is where my very limited signal processing knowledge truly shows. It’s much more visible if I increase the noise level multiplier even higher. Grossly overdone, to show the effect clearly:
I know that the output of simple filters decays gradually (have vague memories of FIR, IIR), and normally one would process a block (say, 16 samples), and get a response that is potentially infinite (IIR). It is my understanding that one should overlap the output from one block with the next, for a FIR filter with a decay of 16:
in[0..15] -> out[0..31] // leave some space for decay
in[16..31] -> take its output, and add out[16..31] from the previous step, keeping 32..47 for the next block
Is that so? I assume that would also apply to 2D FFT.
However, my filter is re-calculated for each block. The measured noise spectrum is constant, but the filter’s response depends on the actual input block:
take the input
pad it to be twice the size in both directions; use the average value as the padding
transform to freq domain
for each freq component f, filter_coefficients(f) = (signal_spectrum(f) - noise_spectrum(f)) / signal_spectrum(f)
then filtered_spectrum = signal_spectrum * filter_coefficients
and padded_filtered_signal = ifft(filtered_spectrum)
take the middle of the padded_filtered_signal (which occupies the space of the original signal block inside the padded square
If the block size is 16x16, process the blocks with 8 pixel overlap:
block with top-left at 0,0
then at 0, 8
then at 0, 16
…
8, 0
8, 8
8, 16
and so on.
The output of each block is masked with a 2D ‘ramp’ that goes from 0 in the corners to 1 in the middle; the masks add up to 1 at all positions.
I’ve changed how I estimate noise level. Previously, I tried to find a “smooth” square, and use its spectrum.
Now, I’ve tried taking some stats (min, max, percentages for each frequency band) collected from many samples taken over the image.