How to create camera noise profiles for darktable

That’s great. Thank you very much! :slight_smile:

Regarding those errors in tools/noise/subr.sh:

This is due to graphics-magick being incompatible with those image-magick operations in subr.sh in line 487 & 494.
As far as I understood they only check if the recorded images are not over/underexposed by calculating some of sort of exposure mean.
Can we achieve that also with graphics-magick, as it has become the default by darktable too, see DefineOptions.cmake#L13?

@asn do you have experience with graphics-magick, or know another way to realize this under/overexposure check?

I wonder whether graphicsmagic provide any substantial benefits over imagemagick. i recently worked with both via flags and they seem for most of darktable uses identical (except for cases til noise profiles where it seems that imagemagick is preferred)

I am not familiar with either, so I don’t have a preference, I only wished that also this script works for both graphics-magick & image-magick, such as the C code.
I find those CLI tools incredibly powerful but at the same time also very complex.
If nobody else feels confident to write down the command(s) for graphics-magick I guess I could sit down during the Chrismas break and find a solution with graphics-magick too then…

I’m having problems capturing appropriate images. For the image below I get the error “_C290293.ORF” not over-exposed (0)". The window is overexposed - in fact it is so blown out that the tree that is normally visible is totally washed out.

As I fiddled with exposure settings I’ve seen values other than (0) for the error code. I’ve searched and have not been able to find a definition for the error codes. Also, I’ve received not under exposed error messages as I adjusted settings on my camera.

Anybody have any suggestions on how I can capture appropriate images?

The window actually does not look blown out. are you shooting manual mode?

How is that overexposed?

Can you please try it again and turn on the live-view histogramm or the overexposure warning in your camera, to confirm you’re actually overexposing?

Maybe follow this tutorial instead?

Based on schenlap’s response that he likes the images from elstoc I figured that I’d try something like that.

The in-camera preview of the raw file had the window as over-exposed. The lack of over exposure may be an artifact of the out of camera jpeg processing.

This photo also gave me an “not over-exposed (0)” error. As you can see DT highlighted the window as over-exposed.

There is no in-camera preview of the raw file, that is the jpeg. You need to over expose by several more stops in order to over expose the raw file.

The photo above is one of a series that I did at various exposures. Some with more and some with less that the DT screenshot that I posted.

Still over exposed if you turn off WB module?

I have some trouble with that how to:
1st: The debain packages in Step 1 don’t have the darktable-gen-noiseprofile script
2nd: Step 1a seem to miss some steps for building :

git clone https://github.com/darktable-org/darktable.git
**cd darktable**
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/opt/darktable -DBUILD_NOISE_TOOLS=ON ..

I do get an error stating to preform git submodule init and git submodule update. After that cmake runs fine.

cd tools/noise 

fails because that directory does not exist, but there is a **…/**tools/noise ?! But that was there before the above cmake command ?!
If I run the darktable-gen-noiseprofile from there instead I get the following error:

[...]
===> Checking profiling RAW images correctness + Jpeg export
--> ISO 100:
/data/Pictures/Olympus_E-M10MarkIV/noise/PC310113.ORF
convert-im6.q16: unable to open image `/data/Pictures/Olympus_E-M10MarkIV/noise/PC310113-large.jpg': Datei oder Verzeichnis nicht gefunden @ error/blob.c/OpenBlob/2874.
convert-im6.q16: no images defined `/data/Pictures/Olympus_E-M10MarkIV/noise/PC310113-thumb.jpg' @ error/convert.c/ConvertImageCommand/3258.

Any help on that? I shoot in raw, so there are no jpg files. I guess the script should create them by itself, right? but why does it fail? I can open the raw-files in darktable without problems.

Not a solution for you, but if you upload your raw files (Dropbox or other filehost) I can try to create the noise profile for you.

Did you use the whole path (/usr/lib/darktable/tools/darktable-gen-noiseprofile) to call the script? This folder is not in the PATH variable, hence it can’t be found without the prefix path.
I’ve successfully run the command on Ubuntu 18.04 and 20.04, with the caveats described in my post above: How to create camera noise profiles for darktable - #99 by da-phil

Is it worth creating new noise profiles for cameras that are already supported? I own two cameras with very similar noise levels, but I always think that the denoise profile module works way better for one of them, while the other always demands way more adjustments to individual parameters. Is it possible to create noise profiles on Mac OS X, for testing?

If you want to play around with noise profiles you can either add a newly calibrated profile to the noiseprofiles.json file of your darktable installation or make a local copy of noiseprofiles.json, add new noise profiles and start darktable with your custom noise profiles:
darktable --noiseprofiles path/to/your/local/noiseprofiles.json

Just give it a go :wink:

As I wrote, this file does not exist in the installation. I run it from the directory cloned via git.

thank you Peter, you can find the files here: Dropbox - Olympus_E-M10MarkIV_noise - Simplify your life