How to create camera noise profiles for darktable

My link posted above is listed as empty when I visit it, though the week for filebin storage is not over. Therefore, I have stored the files again at Filebin | j6g9z6gic4782zr4

Hey sorry, been too busy to process these, I’ll try and get to it soon

Dear Mica,

no problem, actually I feel kind of guilty for not being able to process them myself.
Let me just say that I am impressed with all your contributions, and Darktable rocks!

Both cameras are underestimated and deserve a decent noise profile.
You will make me happy if the profiles should really appear in 3.4 :wink:

Have fun,
Volker

@Colourimeter are all the zips files one set of noise shots each?

Dear Mica,

yes. You see 2 files, one for each camera, that I segmented into 3 parts each to reduce upload errors with filebin. You will have to use an Archive-Program that can read segmented 7zip files - the default for an ubuntu distribution should handle that automatically. Un7zipped, you will find individual shots for each ISO, called ISO100.SRW, ISO125.SRW, ISO160.SRW … and so on. They are accompanied by sidecar files that hold the CC0 license notice (generated with Darktable).

If you prefer another format, just let me know. For me, anything seems easier than processing the noise profiles (there is a clash between my ImageMagick and the profile generation package, and I am not enough of a Linux man to forcefully restore peace between these two. I had a few ideas, but failed in the end).

Have a nice weekend, Volker

Thanks a lot @asn for your great how-to!
I used it to get a denoising profile for my drone (DJI Mavic 2) on Ubuntu 20.04. I succeeded, but wanted to share the bumpy ride:

  1. If you want to install the package darktable-tools-noise on Ubuntu 20.04 you are confronted with the following dependency issue:
The following packages have unmet dependencies:
 graphicsmagick-imagemagick-compat : Conflicts: imagemagick but 8:6.9.10.23+dfsg-2.1ubuntu11.1 is installed
                                     Conflicts: imagemagick:i386 but it is not going to be installed
The following actions will resolve these dependencies:

     Remove the following packages:                                                           
1)     imagemagick [8:6.9.10.23+dfsg-2.1ubuntu11.1 (focal-security, focal-updates, now)]      
2)     imagemagick-6.q16 [8:6.9.10.23+dfsg-2.1ubuntu11.1 (focal-security, focal-updates, now)]

This means that we need to remove imagemagick in favor of the forked graphicsmagick which also provides a compatibilty layer for imagemagick.

  1. Once you installed darktable-tools-noise, you need to remove the check if ! convert --version >/dev/null 2>&1; on line 137 within /usr/lib/darktable/tools/subr.sh. This check somehow seems to only succeed using imagemagick but not with graphicsmagick, which is weird, because we were forced to remove imagemagick in favor of graphicsmagick to begin with :smiley:

  2. While running /usr/lib/darktable/tools/darktable-gen-noiseprofile -d $PWD in the denoising image folder the following warnings popped up:

convert convert: Unrecognized channel type (-channel).
/usr/lib/darktable/tools/subr.sh: line 488: [: : integer expression expected
convert convert: Unrecognized channel type (-channel).
/usr/lib/darktable/tools/subr.sh: line 495: [: : integer expression expected

Nevertheless the profiling succeeded and the results with (wavelet-auto) look convincing :+1:
The noise_result.pdf (400.0 KB) looks quite interesting though.
Here is the final presets.json (1.8 KB) file.

Here is an example: DJI_0751-thumb|690x459, 50%.
I also uploaded my RAW files and the generated profile (whole tar.gz file):
https://filebin.net/j6g9z6gic4782zr4

I also created a PR to get the profile into darktable master: Add Hasselblad L1D-20c (a.k.a. DJI Mavic 2) to noiseprofiles by da-phil · Pull Request #7310 · darktable-org/darktable · GitHub

Please open a darktable bug and contribute the profile so it will be added upstream! Thanks.

Thanks, the profile has been already merged to master: Add Hasselblad L1D-20c (a.k.a. DJI Mavic 2) to noiseprofiles by da-phil · Pull Request #7310 · darktable-org/darktable · GitHub

1 Like

That’s great. Thank you very much! :slight_smile:

Regarding those errors in tools/noise/subr.sh:

This is due to graphics-magick being incompatible with those image-magick operations in subr.sh in line 487 & 494.
As far as I understood they only check if the recorded images are not over/underexposed by calculating some of sort of exposure mean.
Can we achieve that also with graphics-magick, as it has become the default by darktable too, see DefineOptions.cmake#L13?

@asn do you have experience with graphics-magick, or know another way to realize this under/overexposure check?

I wonder whether graphicsmagic provide any substantial benefits over imagemagick. i recently worked with both via flags and they seem for most of darktable uses identical (except for cases til noise profiles where it seems that imagemagick is preferred)

I am not familiar with either, so I don’t have a preference, I only wished that also this script works for both graphics-magick & image-magick, such as the C code.
I find those CLI tools incredibly powerful but at the same time also very complex.
If nobody else feels confident to write down the command(s) for graphics-magick I guess I could sit down during the Chrismas break and find a solution with graphics-magick too then…

I’m having problems capturing appropriate images. For the image below I get the error “_C290293.ORF” not over-exposed (0)". The window is overexposed - in fact it is so blown out that the tree that is normally visible is totally washed out.

As I fiddled with exposure settings I’ve seen values other than (0) for the error code. I’ve searched and have not been able to find a definition for the error codes. Also, I’ve received not under exposed error messages as I adjusted settings on my camera.

Anybody have any suggestions on how I can capture appropriate images?

The window actually does not look blown out. are you shooting manual mode?

How is that overexposed?

Can you please try it again and turn on the live-view histogramm or the overexposure warning in your camera, to confirm you’re actually overexposing?

Maybe follow this tutorial instead?

Based on schenlap’s response that he likes the images from elstoc I figured that I’d try something like that.

The in-camera preview of the raw file had the window as over-exposed. The lack of over exposure may be an artifact of the out of camera jpeg processing.

This photo also gave me an “not over-exposed (0)” error. As you can see DT highlighted the window as over-exposed.

There is no in-camera preview of the raw file, that is the jpeg. You need to over expose by several more stops in order to over expose the raw file.

The photo above is one of a series that I did at various exposures. Some with more and some with less that the DT screenshot that I posted.