Oh, this method so sweet, can’t wait to try it 
I managed to make an image using a combination of outside/inside light that worked quite well. The results however astounded me and now provide an excellent correction (automatically) through the full ISO range of my camera. The only time that I need to adjust my denoise setting is when I push process a part of the image and therefor theoretically modify the image’s local ISO. Profiling took very little time and the developers now include my camera profile in the distribution. If your camera is not currently listed … do it!
2 Likes
Could you explain what advantages the stencil has over the ‘out of focus under- and over-exposed’ approach (like the sample image I posted). Does it give a better quality denoise? I’ve had very good results from the simpler approach but perhaps I’m missing something.
A lot of people tried and had over and underexposed areas, but the results were unusable. You find a lot of open tickets for camera support where it didn’t work. The stencil makes it easier to get good results. We need a good transition from white to black. We are interested in the grey between them. Your image has a lot of color, if you look at mine, there is white, black and grey (no color). The out of focus areas produce the grey values we’re interested in …
1 Like
Based on my unscientific research (I’ve looked through a number of the most recent accepted and rejected profiles in darktable issues) most of the tickets that were rejected seem to have one of two comments:
- Not out of focus enough
- Not enough values between over- and under-exposed
Many of the accepted images have colour in them - indeed white and grey contain all of the colours. I see a lot of accepted profiles based on images similar to mine - an image of a light on a wall in a dark room ensures a good range of values as the brightness drops off gradually with distance and a bare bulb ensures over-exposure.
My understanding is that the reason for defocussing is two-fold. First (most important) it prevents any finely textured objects from masquerading as noise. Second it will (as you say) give more values between over- and under-exposed.
I see where you’re going with the stencil approach though. It definitely could lead to a consistent reproducable way to produce noise profiles, which can only be a good thing and encourage more people to try it.
I thought it might be useful to post some example images that have been successfully used to produce noise profiles (ones that have been accepted into the darktable project). I’ve also provided the relevant issue reference from https://redmine.darktable.org/issues for each image.
As you can see there’s quite a lot of variance in the images but all have produced acceptable profiles.
11842
11865
12007
11684
12095
11496
11291
10213
I’ve recently did the noise profile for my Nikon D5600 following the original darktable post.
https://redmine.darktable.org/issues/12116
No comments yet, so I don’t know if they are going to accept it.
One thing I see you did differently is that you didn’t apply the “raw linear” style to your RAW images.
this makes sure we get linear camera raw rgb with no basecurves, no whitebalance, etc in the way. it is essential that you apply this style.
Just wondering why you choose not to apply it.
Thanks!
When I ran my own, AFAIK the script did all that automatically. Following the original post (and taking into account that the script has changed a bit since then) I just set the camera up, tethered it and ran
~/darktable/tools/noise/gen-profile
The script did the rest - took the images, processed them and auto-generated the tar.gz file
1 Like
I am the one how does most of the darktable noise profile tickets in redmine. Accepting noise profiles is always a subjective process. Always attach the generated tar.gz file (without any modifications), it contains all the information that is required. I first look at noise_result.pdf mainly the “noise_levels” (for good fit) and “flat noise levels” (should be near the ideal line). For noise profiling only the green channel is used. Afterwards I look at the images if they are out of focus. For the images in the initial post i was not sure if I they are good enough. In my view there are better images like the one from elstoc as they have more pixels between over- and underexposed ones. Also profiling images could be more out of focus when using a tele lens, which have a smaller depth of field.
Only profiles from camera with original firmware will be added, also please do not modify presets.json file!
6 Likes
Raw linear style is applied in the script, no need to do anything in darktable before.
2 Likes
I have compiled and installed darktable-2.4.3.tar.xz
I tried to follow the tutorial last night, and everything seemed to run smoothly, but I got an error message after darktable-gen-noiseprofile had processed the raw files.
From the directory I had my raw files in:
$ /opt/darktable/libexec/darktable/tools/darktable-gen-noiseprofile -d .
...
===> Prepare profiling job
--> Remove previous presets
--> Ready to profile images
NOTE: This process takes some time and a lot of memory and disc space
(up-to several gigabytes, depending on the number of ISO settings and
the size of the RAW files.
ERROR: Could not find darktable-noiseprofile tool in /opt/darktable/libexec/darktable/tools
After a while, I grepped after the error message and found it in line 138 in darktable-gen-profile, where there was a check for a file called ‘dt-noiseprofile’, and indeed, no such file existed.
So I made a softlink in /opt/darktable/libexec/darktable/tools :
ln -s darktable-gen-profile dt-noiseprofile
and FINALLY it ran as it should and generated my new noiseprofile.
But, have I missed something in the build-process, or is the check for ‘dt-noiseprofile’ a check from a time when the script actually was called that?
Ouch.
Yes, there is a typo in darktable-gen-noiseprofile in line 137:
- if [ ! -x "$scriptdir/dt-noiseprofile" ]; then
+ if [ ! -x "$scriptdir/darktable-noiseprofile" ]; then
I will create a patch tomorrow.
1 Like
I suspected as much. Thank you for the answer
Hi! Did you have a chance to look at my redmine post for the Nikon D5600? (https://redmine.darktable.org/issues/12116)
Any feedback will be useful
Also due to the lossy after split issue I can’t profile high ISOs files, if I convert the NEF to DNG using Adobe’s DNG Converter, can I profile those DNGs?
PS: I love how FOSS allows us to get in touch with devs and contribute if possible.
1 Like
can you stuff the patch into the graphics:darktable package?
Thanks for the info! I am in the process of picking up a pretty noisy camera (it has other benefits that outweigh the noise) - so when I use DT (hopefully will start with that soon) - I’ll be all set to go thanks to your explanations.
1 Like
I’m struggling with de-noising. I’m using darktable for Windows so can’t go through “Creating noise profiles” steps. Don’t have denoise profile for Panasonic GX7 for ISO 320, 500, 640, 1000, 1250, 1600, 2000, 2500. The 1600 is a basic setting, other are set by camera. Is it possible to upload (or share via cloud) samples and ask somebody to do the job? If this request is acceptable please let me know and I’ll prepare files. Does it make any difference in lens I use for preparing such samples? I have Panasonic lens 14-42/3.5-5.6, 100-300/4.0-5.6, 20/1.7 and Helios 58/2
If you can post the necessary shots, I’d be happy to run the noise profile generation script on them.
1 Like
Could it be beneficial to redo this work if my camera is already supported? The tutorial I saw previously didn’t go to such great lengths (no punch card) and my camera support came from an older firmware (there has been at least a change to the white balance, which I’ve been trying to merge)
Maybe, maybe not. However it would be great to find something which is better than punchcards. I haven’t had the time to experiment again, as I’m working on other tutorials.