How to create camera noise profiles for darktable

da-phil · April 16, 2022, 8:42pm

What are you trying to achieve:

build the whole DT project with noise tools
build only noise tools?

in case of the latter, you can just:

cd tools/noise
mkdir build
cd build
cmake -DCMAKE_INSTALL_LIBEXECDIR=/opt/darktable ..
sudo make install

sugarbravo · April 22, 2022, 5:03pm

I was trying to build darktable and noise tools.

So perhaps the instructions should be updated with separate build within tools/noise.

da-phil · April 22, 2022, 6:07pm

Why did you do cd tools/noise then?
Without that it should work and build the whole darktable project including the noise tools.

sugarbravo · April 22, 2022, 6:37pm

Because darktable was built alright, but tools/noise were not.

da-phil · April 22, 2022, 8:24pm

At least for me it works, just those steps (as you outlined already):

mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX=/opt/darktable -DBUILD_NOISE_TOOLS=ON ..
sudo make install

Somewhere during the build process you should come across this output:

Scanning dependencies of target darktable-noiseprofile
[  9%] Building C object share/darktable/tools/noise/CMakeFiles/darktable-noiseprofile.dir/noiseprofile.c.o

After successfull build & installation you should find the noise calibration tools here: /opt/darktable/libexec/darktable/tools

sugarbravo · April 22, 2022, 10:39pm

Thanks!
Just tried compiling entire darktable once more and I do get /opt/darktable/libexec/darktable/tools/darktable-noiseprofile.

Then what is

mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/opt/darktable -DBUILD_NOISE_TOOLS=ON ..
cd tools/noise
make
sudo make install

for? If compiling tools only, shouldn’t cd tools/noise be the first line?

da-phil · April 23, 2022, 8:48am

Where did you find this line cd tools/noise in the instructions? It does not make any sense after you already changed into the build folder.

If you only want to build the noise tools follow my instructions here: How to create camera noise profiles for darktable - #176 by da-phil

sugarbravo · April 23, 2022, 2:20pm

It’s in this very article we are commenting on, STEP 1a.

da-phil · April 23, 2022, 3:04pm

Ups, I completely forgot about that, I was only thought about README files
Well, then we should ask @asn to update the article before more people fall into the same trap.

paperdigits · April 23, 2022, 4:22pm

Anyone can update an article, just make a pull request against our github.

Peter · May 1, 2022, 12:20pm

First, any manual in how to understand the generated PDF?
noise_result.pdf (337.2 KB)

noise_result.pdf (352.3 KB)

Second, any benefits of using multiple raw files for each ISO? For example like ISO 800 in the print screen.

Third, a thing I found was when I tried to create a noise profile for DJI Air 2. This camera has 12 Mpix and 48 Mpix. I am able to create two noise profiles, but the camera model names are the same for them, so I can’t separate them later in the .json file. Any clue or just to forget it and use one for both?

FalCT60 · July 1, 2022, 3:08pm

Hello,
I think I did not correctly understand the process. It seemed obvious though.
First, one has to open wide the lens at iso 100 (or, I guess, the lowest real one), but still see details, hence the need to eventually decrease the aperture. That, I think I understood.
But, then, one has to make the picture dark - but how ? The only way would be to decrease the aperture at its lowest value. And then I feel totally puzzled.
That sounds not very obvious to me, and this could explain why I could not success so far.
Could someone please explain this to me - and be indulgent - ?
Thanks in advance.

Peter · July 1, 2022, 6:17pm

Large aperture, set the lens to infinity. I used a bright window and with a black curtain beside so that I could get both clipped highlights and crushed shadows.

FalCT60 · July 1, 2022, 6:23pm

So, in fact, just find the correct setting to start from at iso 100, and then just change the speed according to the sensitivity and that’s all.
The part I was refering to is Now you need to set the shutter speed. Make the picture really dark and then make the shutter speed longer till the gap which gives us the white from the monitor is overexposed, pure white see Figure 4. The black around the white paper should be underexposed (pure black).
Maybe some cut/paste leftover…
Thanks.

Peter · July 1, 2022, 6:32pm

Upload a sample and we can check what will need to be done.

Jean-Luc_CECCOLI · July 1, 2022, 8:37pm

Here is the iso 100 picture.
PM4_5446.NEF (32.0 MB)
Rgrds,

Peter · July 1, 2022, 10:12pm

Will take a look tomorrow if no one else does it before me.

Peter · July 2, 2022, 5:36am

Looks very nice, but you should clip the highlights a little bit more, so try the same settings and use a shutter speed of 1/4 sec or 1/2 sec. Use the raw clipping indicator.

FalCT60 · July 2, 2022, 6:32pm

Thanks, I will give it another try when the night is set.
Too much light for now.
However, I still face problems with profiles generation. When I launch the process, I get

jean-luc@e-5530:~/Images/darktable_Noise/D4sNoise$ /opt/darktable/darktable/tools/darktable-gen-noiseprofile -d $(pwd)
===> Check for required tools
--> Check for images handling tools availability
--> Check for images export tools availability
--> Check for profiling tools availability
--> Check for tethering tools availability
--> Check for pdf tools availability

===> Check profiling directory

===> List profiling input RAW images
--> Found ISO 48 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5443.NEF
--> Found ISO 52 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5444.NEF
--> Found ISO 56 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5445.NEF
--> Found ISO 100 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5446.NEF
--> Found ISO 125 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5447.NEF
--> Found ISO 160 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5448.NEF
--> Found ISO 200 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5449.NEF
--> Found ISO 250 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5450.NEF
--> Found ISO 320 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5451.NEF
--> Found ISO 400 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5452.NEF
--> Found ISO 500 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5453.NEF
--> Found ISO 640 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5454.NEF
--> Found ISO 800 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5455.NEF
--> Found ISO 1000 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5456.NEF
--> Found ISO 1250 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5457.NEF
--> Found ISO 1600 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5458.NEF
--> Found ISO 2000 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5459.NEF
--> Found ISO 2500 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5460.NEF
--> Found ISO 3200 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5461.NEF
--> Found ISO 4000 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5462.NEF
--> Found ISO 5000 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5463.NEF
--> Found ISO 6400 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5464.NEF
--> Found ISO 8000 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5465.NEF
--> Found ISO 10000 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5466.NEF
--> Found ISO 12800 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5467.NEF
--> Found ISO 16000 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5468.NEF
--> Found ISO 20000 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5469.NEF
--> Found ISO 25600 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5470.NEF
--> Found ISO 160 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5471.NEF
--> Found ISO 164 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5472.NEF
--> Found ISO 168 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5473.NEF
--> Found ISO 180 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5474.NEF
--> Found ISO 192 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5475.NEF
--> Found ISO 204 image: /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5476.NEF

The script will use existing input RAW images for the profiling. No more
shot will be taken.

===> Checking profiling RAW images correctness + Jpeg export
--> ISO 48:
    /home/jean-luc/Images/darktable_Noise/D4sNoise/PM4_5443.NEF
usage: darktable-cli [options] [IMG_1234.{RAW,..}|image_folder/]

options:

  --cachedir <user cache directory>
  --conf <key>=<value>
  --configdir <user config directory>
  -d {all,cache,camctl,camsupport,control,dev,fswatch,imageio,input,
      ioporder,lighttable,lua,masks,memory,nan,opencl,params,perf,demosaic
      pwstorage,print,signal,sql,undo,act_on}
  --d-signal <signal> 
  --d-signal-act <all,raise,connect,disconnect,print-trace>
  --datadir <data directory>
  --enforce-tiling
  --disable-opencl
  -h, --help
  --library <library file>
  --localedir <locale directory>
  --moduledir <module directory>
  --noiseprofiles <noiseprofiles json file>
  -t <num openmp threads>
  --tmpdir <tmp directory>
  --version
jean-luc@e-5530:~/Images/darktable_Noise/D4sNoise$

I tried with the script within my dt build, then building using your explanations, but still the same result.
So I am sort of disappointed.

Peter · July 2, 2022, 7:16pm

Upload all raw files to Google drive or Dropbox. Your ISO 100 raw file worked for me to process.