How to create camera noise profiles for darktable

It’s in this very article we are commenting on, STEP 1a.

Ups, I completely forgot about that, I was only thought about README files :smiley:
Well, then we should ask @asn to update the article before more people fall into the same trap.

Anyone can update an article, just make a pull request against our github.

First, any manual in how to understand the generated PDF?
noise_result.pdf (337.2 KB)

noise_result.pdf (352.3 KB)

Second, any benefits of using multiple raw files for each ISO? For example like ISO 800 in the print screen.

Third, a thing I found was when I tried to create a noise profile for DJI Air 2. This camera has 12 Mpix and 48 Mpix. I am able to create two noise profiles, but the camera model names are the same for them, so I can’t separate them later in the .json file. Any clue or just to forget it and use one for both?

Hello,
I think I did not correctly understand the process. It seemed obvious though.
First, one has to open wide the lens at iso 100 (or, I guess, the lowest real one), but still see details, hence the need to eventually decrease the aperture. That, I think I understood.
But, then, one has to make the picture dark - but how ? The only way would be to decrease the aperture at its lowest value. And then I feel totally puzzled.
That sounds not very obvious to me, and this could explain why I could not success so far.
Could someone please explain this to me - and be indulgent - ?
Thanks in advance.

Large aperture, set the lens to infinity. I used a bright window and with a black curtain beside so that I could get both clipped highlights and crushed shadows.

So, in fact, just find the correct setting to start from at iso 100, and then just change the speed according to the sensitivity and that’s all.
The part I was refering to is Now you need to set the shutter speed. Make the picture really dark and then make the shutter speed longer till the gap which gives us the white from the monitor is overexposed, pure white see Figure 4. The black around the white paper should be underexposed (pure black).
Maybe some cut/paste leftover…
Thanks.

Upload a sample and we can check what will need to be done.

Here is the iso 100 picture.
PM4_5446.NEF (32.0 MB)
Rgrds,

Will take a look tomorrow if no one else does it before me.

Looks very nice, but you should clip the highlights a little bit more, so try the same settings and use a shutter speed of 1/4 sec or 1/2 sec. Use the raw clipping indicator.

Thanks, I will give it another try when the night is set.
Too much light for now.
However, I still face problems with profiles generation. When I launch the process, I get

jean-luc@e-5530:~/Images/darktable_Noise/D4sNoise$ /opt/darktable/darktable/tools/darktable-gen-noiseprofile -d $(pwd)
===> Check for required tools
--> Check for images handling tools availability
--> Check for images export tools availability
--> Check for profiling tools availability
--> Check for tethering tools availability
--> Check for pdf tools availability

===> Check profiling directory

===> List profiling input RAW images
--> Found ISO 48 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5443.NEF
--> Found ISO 52 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5444.NEF
--> Found ISO 56 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5445.NEF
--> Found ISO 100 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5446.NEF
--> Found ISO 125 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5447.NEF
--> Found ISO 160 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5448.NEF
--> Found ISO 200 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5449.NEF
--> Found ISO 250 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5450.NEF
--> Found ISO 320 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5451.NEF
--> Found ISO 400 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5452.NEF
--> Found ISO 500 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5453.NEF
--> Found ISO 640 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5454.NEF
--> Found ISO 800 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5455.NEF
--> Found ISO 1000 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5456.NEF
--> Found ISO 1250 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5457.NEF
--> Found ISO 1600 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5458.NEF
--> Found ISO 2000 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5459.NEF
--> Found ISO 2500 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5460.NEF
--> Found ISO 3200 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5461.NEF
--> Found ISO 4000 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5462.NEF
--> Found ISO 5000 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5463.NEF
--> Found ISO 6400 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5464.NEF
--> Found ISO 8000 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5465.NEF
--> Found ISO 10000 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5466.NEF
--> Found ISO 12800 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5467.NEF
--> Found ISO 16000 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5468.NEF
--> Found ISO 20000 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5469.NEF
--> Found ISO 25600 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5470.NEF
--> Found ISO 160 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5471.NEF
--> Found ISO 164 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5472.NEF
--> Found ISO 168 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5473.NEF
--> Found ISO 180 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5474.NEF
--> Found ISO 192 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5475.NEF
--> Found ISO 204 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5476.NEF

The script will use existing input RAW images for the profiling. No more
shot will be taken.

===> Checking profiling RAW images correctness + Jpeg export
--> ISO 48:
    /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5443.NEF
usage: darktable-cli [options] [IMG_1234.{RAW,..}|image_folder/]

options:

  --cachedir <user cache directory>
  --conf <key>=<value>
  --configdir <user config directory>
  -d {all,cache,camctl,camsupport,control,dev,fswatch,imageio,input,
      ioporder,lighttable,lua,masks,memory,nan,opencl,params,perf,demosaic
      pwstorage,print,signal,sql,undo,act_on}
  --d-signal <signal> 
  --d-signal-act <all,raise,connect,disconnect,print-trace>
  --datadir <data directory>
  --enforce-tiling
  --disable-opencl
  -h, --help
  --library <library file>
  --localedir <locale directory>
  --moduledir <module directory>
  --noiseprofiles <noiseprofiles json file>
  -t <num openmp threads>
  --tmpdir <tmp directory>
  --version
jean-luc@e-5530:~/Images/darktable_Noise/D4sNoise$ 

I tried with the script within my dt build, then building using your explanations, but still the same result.
So I am sort of disappointed.

Upload all raw files to Google drive or Dropbox. Your ISO 100 raw file worked for me to process.

Sorry it took so long. I had to face a few weird problems. All solved.
Here is the link to the photos.
The first three are iso 50, 64 & 80, and the last six are the ones over 25600 - they are really very noisy !
I did not know if they were useful or not.
Thanks for your help.

Will take a look tomorrow.

2 Likes

Here you are. Create a new issue here and upload the embedded file Sign in to GitHub · GitHub

darktable-noiseprofile-20220703.tar.gz (10.2 MB)

Many thanks, Peter. :clap:
The issue was filed a couple of minutes ago.
I intend to give it another try when v4.0 is available in repos and, if it fails once more, I then will open an issue for this.
Best regards.

When I run the denoise profile generator tool with darktable 4.0 I get the same terminal output as @FalCT60

===> Check for required tools
--> Check for images handling tools availability
--> Check for images export tools availability
--> Check for profiling tools availability
--> Check for tethering tools availability
--> Check for pdf tools availability

===> Check profiling directory

===> List profiling input RAW images
--> Found ISO 100 image: /home/user/Pictures/IMG_0568.CR3
--> Found ISO 200 image: /home/user/Pictures/IMG_0569.CR3
--> Found ISO 400 image: /home/user/Pictures/IMG_0570.CR3
--> Found ISO 800 image: /home/user/Pictures/IMG_0571.CR3
--> Found ISO 1600 image: /home/user/Pictures/IMG_0572.CR3
--> Found ISO 3200 image: /home/user/Pictures/IMG_0573.CR3
--> Found ISO 6400 image: /home/user/Pictures/IMG_0574.CR3
--> Found ISO 12800 image: /home/user/Pictures/IMG_0575.CR3
--> Found ISO 25600 image: /home/user/Pictures/IMG_0576.CR3

===> Checking profiling RAW images correctness + Jpeg export
--> ISO 100:
    /home/user/Pictures/IMG_0568.CR3
usage: darktable-cli [options] [IMG_1234.{RAW,..}|image_folder/]

options:

  --cachedir <user cache directory>
  --conf <key>=<value>
  --configdir <user config directory>
  -d {all,cache,camctl,camsupport,control,dev,fswatch,imageio,input,
      ioporder,lighttable,lua,masks,memory,nan,opencl,params,perf,demosaic
      pwstorage,print,signal,sql,undo,act_on,tiling,verbose}
  --d-signal <signal> 
  --d-signal-act <all,raise,connect,disconnect,print-trace>
  --datadir <data directory>
  --disable-opencl
  -h, --help
  --library <library file>
  --localedir <locale directory>
  --luacmd <lua command>
  --moduledir <module directory>
  --noiseprofiles <noiseprofiles json file>
  -t <num openmp threads>
  --tmpdir <tmp directory>
  --version

I’ve run the tool in Fedora, Manjaro, and Debian-testing and get the same result everytime. Does anyone have any suggestions on how to resolve this? Is it possible the denoise tool is feeding commands into darktable-cli with incorrect syntax?

The raw data should be the same if I convert them from CR3 to DNG and then create a denoise profile from them. I don’t have CR3 support in my VirtualBox darktable version. If you want you can send me the files and I can try generate a profile. You can try the profile first before sending it in to Github.

Thanks Peter, I will take you up on that offer, but want to reshoot the pictures first to make sure they’re clipping on all three channels. I will send a link tonight.