How to create camera noise profiles for darktable

Most likely not, I can’t even see any noise at ISO 50 that warrants a denoise pass anyways, I’m just trying to get as full of a dataset as possible to send for inclusion into darktable. Could probably let interpolation handle the Lo extensions if I needed to…

The Hi extended ISOs would definitely be an issue in providing said dataset, as they are being detected as very incorrect values than what they should be…

@rawfiner, any clues?

At minimum, I’ll probably fire off a bug report for this too like how you mentioned, so that it can be fixed in a future release.

I’m sure there will be at least one other person that runs into this eventually :sweat_smile:.

ETA: https://github.com/darktable-org/darktable/issues/12264

Unfortunately the script sometimes badly reads the ISO values. It is maybe a similar issue as this one: https://github.com/darktable-org/darktable/issues/8992
The good news is that the ISO values can be manually corrected in the noiseprofile json file that the script outputs in order to match the values showed by darktable (in darkroom or lighttables). That way, darktable will correctly detect the noise profile.

Ya, I’ve done that for my local copy of darktable and it works fine there, but I want to be able to submit the profile for inclusion too and I’m not sure if having to modify the json file to correct these ISOs would be accepted for that since it would no longer match up with the generated pdfs…

Yes it will be accepted even if some ISO values have been corrected manually, you can submit it. All that matters is that the curves looks good, and that darktable correctly sets the profile in denoise profile module.

Ah ok. I’ll submit mine later today with the hopes if it being accepted in that case, just with a note stating the ISO changes to try and prevent confusion.

Hi,
I got an error during the compilation process.

-- Found msgfmt to convert .po file. Translation enabled
CMake Error at src/external/CMakeLists.txt:2 (add_subdirectory):
The source directory
/home/jacotux/tmpdir/darktable/src/external/libxcf
does not contain a CMakeLists.txt file.
CMake Error at src/external/CMakeLists.txt:18 (message):
RawSpeed submodule not found.  You probably want to run:
$ git submodule init
and then
$ git submodule update
-- Configuring incomplete, errors occurred!

any idea how to fix it, thanks

Hi @JacoTux, and welcome!

First, please try what the machine tells you to do:

git submodule init
git submodule update

Have fun!
Claes in Lund, Sweden

:face_with_hand_over_mouth: Roooh, what an idiot I am, of course.

Thank you Claes.

Yes, I ran into it as well. I proposed a solution in that issue: darktable-gen-noiseprofile incorrectly reads extended ISO settings from Z 6ii · Issue #12264 · darktable-org/darktable · GitHub. It seems that exiv2 returns a bad value for low/high isos using the existing cli switches in file subr.sh.

I had also issue with the generation of the noise_result.pdf that was produced using pdftk. This is in my Ubuntu 22.04 installed using snap and if the raw files are on symlinked drive, pdftk was not able to find the files. The solution is either here: java - Why is PDFTK saying it can't find files that are right there? - Unix & Linux Stack Exchange or in subr.sh - checking for gs first and only later for pdftk.

1 Like

Hi,
For those interested, I created a video on how to check the noiseprofiles:

9 Likes

Finally got this going on openSUSE Tumbleweed, running the tool, and getting the following error:

===> Checking profiling RAW images correctness + Jpeg export
--> ISO 100:
    /home/zingerpb/Pictures/NoisePhotos/DSC01496.ARW
RawSpeed:Unable to find camera in database: 'SONY' 'ILCE-6700' ''
Please consider providing samples on <https://raw.pixls.us/>, thanks!
     0.4646 [rawspeed] (DSC01496.ARW) bool rawspeed::RawDecoder::checkCameraSupported(const rawspeed::CameraMetaData*, const std::string&, const std::string&, const std::string&), line 171: Camera 'SONY' 'ILCE-6700', mode '' not supported, and not allowed to guess. Sorry.
     0.4646 [libraw_open] extensions whitelist: `cr3'
     0.4665 [temperature] failed to read camera white balance information from `DSC01496.ARW'!
     0.4682 [dt_imageio_export_with_flags] mipmap allocation for `/home/zingerpb/Pictures/NoisePhotos/DSC01496-large.jpg' failed
     0.4683 [imageio_storage_disk] could not export to file: `/home/zingerpb/Pictures/NoisePhotos/DSC01496-large.jpg'!

However, I know the camera profile has been updated and included in the latest DT release, as well as being on rawspeed.

6700 added 17 August 2023. darktable 4.4.2 was released earlier, wasn’t it?

Have you checked your local cameras.xml that it has 6700? If not, just copy paste from here

Thanks Peter, that helped. The noiseprofile tool was pulling from a native install of darktable which didn’t have that profile, but I’ve been using the flatpak (to maintain consistency between two computers/os installs). I was able to copy - paste the cameras.xml file and that fixed it.

1 Like

I’m planning to create a noise profile for the Nikon Z8, which still seem to be missing. I tried printing out a gradient on my printer, which is an (Epson 7750) inkjet printer, on normal paper.

However, the dark parts are not dark enough and with backlight, exposing so that the white part burns out, the darkest part of the gradient is perhaps only mid-gray or so.

Have anyone else printed gradients on inkjet printers? Or is it better to use laser printer?

I just used a black curtain, overcast sky and a window.

OK, thanks – that sounds manageable then. I was assuming I would need to create an as even as possible gradient from dark to overexposed, but what you did sounds more like you relied on lens blurring to create the gradient? The only problem here is that are lots of clouds currently, so I cannot focus on infinity – then I’ll get sharp images of clouds :slight_smile:

Then focus on the closest distance you can without getting any details.

1 Like

Hi @rawfiner, I am gonna use OLED screen to generate the profile. How was your gradient figure generated?

Since the provided figure isn’t comes with the black background and white part, I want to generate the gradient by myself.

Is it a CIE nonlinear gradient or a linear gradient?

Thanks!