Before demosaicing - Color noise sensor calibration / profiling. Is this a good idea, or can we skip it?

Software darktable
1

Hello everyone,

First of all, I’d like to thank all the developers who have put so much time and know-how into Darktable. Unfortunately, I am not a programmer, nor do I have any expertise in color theory, etc. But since Darktable is an open-source project, I wanted to share an idea that has been on my mind for a while, and based on my research, it doesn’t seem to exist yet.

The main idea is to average the individual R+G+B pixels separately, before the demosaic process, statistically across the respective ISO ranges, in order to achieve significantly better color consistency and reduce color noise. The module should also be designed so that users can apply it to their own camera in Darktable.

My (naive) idea:
You photograph (ideally without a lens) a gray card or gradient card multiple times for a given ISO range (e.g., ISO 800).
Since the resolution and the distribution of the RGB pixels on the sensor and the “exposure” via the gray card are known (RAW), you can now statistically average each “color channel” separately on a pixel level. Due to the gray card, the structure of the image is also not altered.

This means, for example, in the red channel, you would only average the red pixels, while all non-red pixels (G+B) in the array are deleted. This should theoretically give you a statistically clean red channel on the pixel level. The same would be done for blue and green. Only after averaging would the demosaic process take place.

In my view, this would effectively prevent color channel cross-talk and result in significantly improved color consistency (no color spots in the blue sky). Theoretically, this should even have the effect of improved luminance noise. I also think that many older cameras could benefit greatly from this. It could also be used to perfectly profile the vignetting of lenses…

As I mentioned… I’m not the right person for technical discussions, but maybe you can make something of this idea. :slight_smile:

I don’t speak English…Translate using ChatGPT

sensor denoise
sensor denoise1000×375 55.4 KB

Hallo leute.

Ersteinmal vielen dank an die ganzen Programmierer welche hier soviel zeit und know-how in Darktable einfließen lassen. Ich bin leider kein Programmierer und habe auch kein Wissen in Farblehre etc…

Aber da Darktable ein offenes Projekt ist wollte ich mal eine Idee einbringen, welche ich schon länger in meinem Kopf habe und nach meiner Recherche so noch nicht zu existieren scheint.

Mein Grundgedanke ist dabei, dass man die Pixel R+G+B jeweils getrennt, vor dem demosaic, statistisch über die jeweiligen isobereiche mittelt, um eine deutlich bessere Farbkonsistenz und weniger Farbrauschen zu erzeugen. Auch sollte das Modul so angelegt sein dass man es in darktable für seine eigene Kamera anwenden kann.

Mine naive Idee:

man fotografiert (ideal ohne objektiv) mehrfach eine Graukarte / Grauverlaufskarte für einen isobereich (z.B iso 800).

Da die Auflösung und die Verteilung der RGB-pixel des Sensors und die „Belichtung“ durch die Graukarte bekannt sind (RAW), kann man jetzt jeweils die „Farbkanäle“ einzeln, auf Pixelebene, statistisch farblich mitteln. Aufrund der Graukarte greift man auch nicht in die Struktur des Bildes ein.

Das heißt, man mittelt statistisch im Rotkanal nur die roten Pixel und alle nicht dazugehörigen Pixel (G+B) im Array werden gelöscht. Dadurch erhält man theoretisch auf Pixelebene ein statistisch sauberen Rotkanal. Das gleiche würde mit Blau und Grün gemacht. Erst jetzt nach der Mittelung würde das demosaic stattfinden.

Dadurch würde, nach meiner Vorstellung, das Kanalübersprechen effektiv verhindert und eine deutlich verbesserte Farbkonsistenz erreicht (keine Farbflecken im blauen Himmel). Theoretisch müsste sogar der Effekt eines verbesserten Luminazrauschen auftreten. Ich denke auch das viele ältere kameras davon sehr profitieren könnten. Auch könnte man damit die Vignetierung von objektiven perfekt profilieren……

wie gesagt…ich bin technisch nicht der richtige Gesprächspartner aber vielleicht könnt mit der idee etwas anfangen :slight_smile:

2

It’s very very easy to remove noise.

The hard part is to remove noise without removing detail.

3

have a look at darktable user manual - raw denoise

4

It may be possible to simply remove color noise. However, the problem is the resulting artifacts, such as purple blotches in a blue sky. (In my experience, this is very common with Sony 6000 series models; the D750 also had similar issues at times.)

While you do reduce the color noise into a kind of “homogeneous mass,” you don’t actually achieve color neutrality across the surface. That’s why my idea is to profile all pixels per channel using a gray card “kind of like an Excel spreadsheet where the values in each cell are adjusted. Before: 100, 85, 70, 120, 50… After: 90, 95, 88, 92, 91…” , as a kind of correction mask for the individual RGB pixels.

My intention is not to correct luminance noise (details); that would just be a side effect that might occur.

I’m familiar with and have read the article on RAW denoising. If I understand correctly, it adjusts the respective contribution in the channels via “curves.” In my understanding, this only results in a global amplification or reduction within the respective channel. As described, this does not create uniformity in color reproduction. And as ISO increases, the problem is known to become more pronounced.
:slight_smile:

5

Hallo @Suki2019

Is your noise consistent on a per-pixel basis from shot to shot?

I don’t know Sony cameras, but perhaps there is a setting to record dark noise to help correct raw profiles. Such a procedure might include recording sensor data with the lens cap on.

Noise has been studied extensively for CMOS imaging sensors, well beyond the dt community. You might be interested in learning about learning from astronomical photography. Noise removal techniques are part of Siril, for example. With static images, averaging over several shots helps, as does background subtraction.

For a general article see e.g. Sensor cal and color

With dt specifically, there is a video from @rawfiner that is excellent; Denoising with dt 2.6

It’s an old video, and the interface may have been updated, but the content of the video is “gold.”

I’m not a camera-noise expert, but I do know a little about the sensors. I can say with confidence that Sony’s sensor engineering and process technology are world-class, which may be why most of the world uses Sony imaging sensors.

1 Like
6

That’s a good video that I have often shared. The bilateral denoise is now called surface blur in DT and with careful tweaking as shown in the video it can be quite effective

1 Like
7

Hello Douglas,

noise is never consistent—that much I’m aware of. However, with cameras that exhibit certain “color errors,” the areas where these occur tend to be the same. (Unfortunately, I don’t have any example photos at the moment (private).) The video you mentioned is very interesting and demonstrates quite well, using the ISO 51,000 example image, what I’m essentially trying to avoid or at least significantly reduce from the outset.

These extreme false colors that can appear at higher ISO levels could, in my view, largely be reduced with the approach I have in mind, by applying a kind of correction matrix that evens out the values. Not in the sense that all pixels would have the same value (like a gray card), but rather so that, for example, an array of 20 pixels would have relatively similar values across the sensor area.

As shown, more and more errors occur in sensors as ISO increases—either due to the AD converter or the photodiodes themselves (amplification), or both. However, each sensor unit tends to behave similarly in this regard. In astrophotography, it also seems common practice to try to eliminate luminance noise and color noise (color shifts) by combining multiple exposures. However, this is not done at the pixel level of the sensors. And in darktable, there is no “accessible” module for that either.

8

This sounds like something that would be calibrated externally, like using a colour checker card in Colour Calibration module?

9

Ok, thank you.

I lack the experience to say whether the scheme you propose would work well for “fixed patterned noise” as discussed in the article. Others would need to comment.

To my understanding, with modern cameras high-iso noise is dominated by sensor noise (dark current, photon noise etc.*). So personally my approach is to try to collect more light so that lower iso is sufficient. With my MFT camera I’ve set an upper iso limit of 3200. Larger sensors generally have better low-light performance.

  • (There are some fine details related to signal conversion and amplification, and dual-amp cameras exist to help bridge high and low iso conditions. But I think that’s more of a detail than the core issue.)
10

Yes—just that this type of calibration (grey card) takes place before demosaicing, so that color shifts and false colors caused by readout errors or amplification (ISO) are minimized.