How to create camera noise profiles for darktable
An easy way to create correct profiling pictures
Noise in digital images is similar to film grain in analogue photography. In digital cameras, noise is either created by the amplification of digital signals or heat produced by the sensor. It appears as random, colored speckles on an otherwise smooth surface and can significantly degrade image quality.
Noise is always present, and if it gets too pronounced, it detracts from the image and needs to be mitigated. Removing noise can decrease image quality or sharpness. There are different algorithms to reduce noise, but the best option is if having profiles for a camera to understand the noise patterns a camera model produces.
Noise reduction is an image restoration process. You want to remove the digital artefacts from the image in such a way that the original image is discernible. These artefacts can be just some kind of grain (luminance noise) or colorful, disturbing dots (chroma noise). It can either add to a picture or detract from it. If the noise is disturbing, we want to remove it. The following pictures show a picture with noise and a denoised version:
To get the best noise reduction, we need to generate noise profiles for each ISO value for a camera.
For every ISO value your camera has, you have to take a picture. The pictures need to be exposed a particular way to gather the information correctly. The photos need to be out of focus with a widespread histogram like in the following image:
We need overexposed and underexposed areas, but mostly particularly the grey areas in between. These areas contain the information we are looking for.
Let’s go through the noise profile generation step by step. For easier creation of the required pictures, we will create a stencil which will make it easier to capture the photos.
You need to get some thicker black paper or cardboard. No light should shine through it! First we need to use the lens hood to get the size. The lens hood helps to move the paper away from the lens a bit and the lens hood gives us something to attach it to. Then we need to create a punch card. For wide angle lenses you need a close raster and for longer focal lengths, a wider raster. It is harder to create it for compact cameras with small lenses (check below).
Find the middle and mark the size of the lens hood:
If you have the size, draw a grid on the paper:
Once you have done that you need to choose a punch card raster for your focal length. I use a 16mm wide angle lens on a full frame body, so I choose a raster with a lot of holes:
Untested: For a 50mm or 85mm lens I think you should start with 5 holes in the middle created just with a needle. Put your stencil on the lens hood and check. Then you know if you need bigger holes and maybe how much. Please share your findings in the comments below!
I guess you would create a stencil, like for bigger lenses, but create a funnel to the camera. Contributions and ideas are welcome!
Wait for a cloudy day with thick clouds and no sun to take the pictures. The problem is the shutter speed and it is likely that you’ll hit the limit. My camera has 37 ISO values (including extended iso), so I need to start with 0.6 seconds exposure time to take the last picture with the limit of my camera, 1/8000 of a second exposure time. So a darker day helps to start with a slow shutter speed.
Use a tripod and point the camera to the sky, attach the lens hood and put the punch card on it. Better make sure that all filters are removed, so we don’t get any strange artefacts. In the end the setup should look like this:
Choose the fastest aperture available on your lens (e.g. f/2.8 or even faster), change the camera to manual focus, and focus on infinity. Take the shot! The result should look like this:
The holes will overexpose the picture, but you also need an underexposed area. So start to put most of my dark areas in the middle of the histogram and moved it to the black (left) side of the histogram until the first values start to clip. It is important to not to clip to much, as we are mostly interested the grey values between the overexposed and underexposed areas.
Once you’re done taking the pictures it is time to move to the computer.
If this gives you the help of the tool, continue with STEP 2 othersise go to STEP 1a
Your darktable installation doesn’t offer the noise tools so you need to compile it yourself. Before you start make sure that you have the following dependencies installed on your system:
- convert (ImageMagick)
Get the darktable source code using git:
git clone https://github.com/darktable-org/darktable.git
Now change to the source and build the tools for creating noise profiles using:
mkdir build cd build cmake -DCMAKE_INSTALL_PREFIX=/opt/darktable -DBUILD_NOISE_TOOLS=ON .. cd tools/noise make sudo make install
Download the pictures from your camera and change to the directory on the commandline:
and run the following command:
/usr/lib/darktable/tools/darktable-gen-noiseprofile -d $(pwd)
or if you had to download and build the source, run:
/opt/darktable_source/lib/tools/darktable-gen-noiseprofile -d $(pwd)
This will automatically do everything for you. Note that this can take quite some time to finish. I think it took 15 to 20 minutes on my machine. If a picture is not shot correctly, the tool will tell you the image name and you have to recapture the picture with that ISO.
The tool will tell you, once completed, how to test and verify the noise profiles you created.
Once the tool finished, you end up with a tarball you can send to darktable for inclusion. You can open a bug at:
The interesting files are the
presets.json file (darktable input) and, for the
developers, the noise_result.pdf file. You can find an example PDF
here. It is a
collection of diagrams showing the histogram for each picture and the results
of the calculations.
A detailed explanation of the diagrams and the math behind it can be found in the original noise profile tutorial by Johannes Hanika.
I’ve created the stencil above to make it easier to create noise profiles. However I’ve tried different ways to create the profiles and here is one which was a good idea but failed for low ISO values (ISO <= 320). We are in the open source world, and I think it is important to share failures too. Others may have an idea to improve it or at least learn from it.
For a simpler approach than the one described above, I’ve created a gradient from black to white. Then I used some black cardboard to attached it to the monitor to get some real black. Remember you need an underexposed area and the monitor is not able to output real black, as it is backlit.
In the end my setup looked liked this:
I’ve turned off the lights and took the shots. However the results for ISO values below and equal to ISO320 are not good. All other ISO values looked fine.
If you’re interested in the results, you can find them here:
Please also share pictures of working stencils you created.
Feedback is very much welcome in the comments below!
This is a companion discussion topic for the original entry at https://pixls.us/articles/how-to-create-camera-noise-profiles-for-darktable/