James_Wyper
(James Wyper)
January 15, 2022, 9:44am
#163
3.6.1 gives the same results so it must be something specific to my build or environment. As you’ve kindly created the profiles I’ll let this rest now. Thank you again Peter.
1 Like
Using darktable master or 3.8.x, same problem:
```
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/opt/darktable -DBUILD_NOISE_TOOLS=ON ..
cd tools/noise
make
sudo make install
```
cd tools/noise
should be preceded by cd ..
the following make
command produces make: *** No targets specified and no makefile found. Stop.
Trying with debian package darktable-tools-noise
: during install removes imagemagick for graphicsmagick-imagemagick-compat
, but when I run (ubuntu 20.04)
$ /usr/lib/darktable/tools/darktable-gen-noiseprofile
===> Check for required tools
--> Check for images handling tools availability
--> Check for images export tools availability
ImageMagick is required to check input images correctness. Please
install this package and re-run this script.
NOTE: You may have to remove GraphicsMagick-related packages before.
--> Check for profiling tools availability
--> Check for tethering tools availability
--> Check for pdf tools availability
i.e. it wants imagemagick!!! if I install it, /usr/lib/darktable/tools/darktable-gen-noiseprofile has desappeard because it was incompatible with imagemagick.
I see that I have a ~/git/darktable/tools/noise/darktable-gen-noiseprofile
(what version? it doesn’t report it!), using it, whatever the exposition of the raw file I get the "~/raw/paraOrganizar/IMG_9712.CR3" not over-exposed (0)
error and I cannot generate the noise profile
da-phil
February 6, 2022, 5:23pm
#165
Regarding the imagemagick / graphicsmagick, there is already an issue reported on github:
opened 11:16AM - 03 Jan 21 UTC
scope: software support
**Describe the bug**
This has been already mentioned in the ["How to create cam… era noise profiles for darktable" article on discuss.pixls.us](https://discuss.pixls.us/t/how-to-create-camera-noise-profiles-for-darktable/7428/99).
When installing `darktable` on Ubuntu 18.04 / 20.04,` imagemagick` is used as dependency [see also DefineOptions.cmake#L13](https://github.com/darktable-org/darktable/blob/master/DefineOptions.cmake#L13).
However when installing the `darktable-tools-noise` the forked alternative `graphicsmagick-imagemagick` is required for installation and leads to `imagemagick` to be uninstalled, see
```bash
The following NEW packages will be installed:
darktable-tools-noise graphicsmagick{a} graphicsmagick-imagemagick-compat{ab}
0 packages upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
Need to get 680 kB of archives. After unpacking 5.396 kB will be used.
The following packages have unmet dependencies:
graphicsmagick-imagemagick-compat : Conflicts: imagemagick but it is not going to be installed
Conflicts: imagemagick:i386 but it is not going to be installed
The following actions will resolve these dependencies:
Remove the following packages:
1) imagemagick-6.q16 [8:6.9.10.23+dfsg-2.1ubuntu11.2 (focal-security, focal-updates, now)]
```
The culprit here lies in [`tools/noise/subr.sh` in line 487 & 494](https://github.com/darktable-org/darktable/blob/master/tools/noise/subr.sh#L487) in which very a very imagemagick specifc command is used to detect whether the noise calibration images are neither over- nor under-exposed. This command however is not compatible with `graphicsmagick-imagemagick`.
It gives the following error output during a noise profile generation run:
```
convert convert: Unrecognized channel type (-channel).
/usr/lib/darktable/tools/subr.sh: line 488: [: : integer expression expected
convert convert: Unrecognized channel type (-channel).
/usr/lib/darktable/tools/subr.sh: line 495: [: : integer expression expected
```
**Expected behavior**
Because the darktables C code seems to be compatible to both imagemagick and graphicsmagick APIs I'd assume the same for the scripts which use the CLI interface of the aforementioned tool.
I'd expect the script to use CLI calls which are either compatible with both tools, or just have a code path for both tools.
**Platform (please complete the following information):**
- Darktable Version: 3.2.1 and 3.4.0
- OS: Ubuntu 18.04 / 20.04
I also think that the script needs to be updated.
What errors are you getting and why do you think it needs to be updated?
da-phil
February 6, 2022, 5:42pm
#167
see comments from @rawfiner and me in the github issue.
In a nutshell: script only works and gives intended results with imagemagick, not with graphicsmagick, although the darktable C code works with both C APIs.
As far as I remember, at least the debian packages have darktable depend on grahpicsmagick and this breaks the noise profiling script.
IMHO the “sufficiently over- and underexposed” checks could be simplified.
If you haven’t contacted the packager, then it isn’t surprising that it hasn’t been resolved. Packaging is handled by the community.
da-phil
February 6, 2022, 6:01pm
#169
That’s not the problem, the script is the problem, it should - such as the C code - be compatible with both imagemagick & graphicsmagick.
da-phil
February 8, 2022, 8:36pm
#170
I took some time to address the problem in the following PR, please have a look and leave feedback:
darktable-org:master
← da-phil:pl/make_noise_profile_script_compatible_to_graphicsmagick
opened 08:09PM - 08 Feb 22 UTC
Currently the subroutine to check whether noise calibration images uses a very c… omplicated set of ImageMagick parameters which is not compatible with GraphicsImage.
This PR adds parameter-sets which work for both tools.
Problem statement and explanation of the solution can be found here: https://github.com/darktable-org/darktable/issues/7651#issuecomment-1032911519
Fixes https://github.com/darktable-org/darktable/issues/7651
da-phil
February 8, 2022, 8:42pm
#171
Paolo_Benvenuto:
I see that I have a ~/git/darktable/tools/noise/darktable-gen-noiseprofile
(what version? it doesn’t report it!), using it, whatever the exposition of the raw file I get the "~/raw/paraOrganizar/IMG_9712.CR3" not over-exposed (0)
error and I cannot generate the noise profile
As already asked on github, can you please share the file(s), then I’ll have a look.
sugarbravo
(Šarūnas Burdulis)
April 1, 2022, 4:14pm
#172
The script uses darktable-cli generated JPEG files to check for under/over-exposure, and makes wrong conclusions about under/over-exposure in RAW.
darix
(darix)
April 1, 2022, 4:18pm
#173
Please file a bug and explain the details of your findings.
This has already been raised by @Paolo_Benvenuto here:
opened 02:56AM - 14 Mar 22 UTC
closed 09:15AM - 16 Mar 22 UTC
master
I'm trying to follow `https://pixls.us/articles/how-to-create-camera-n… oise-profiles-for-darktable/` in order to generate a noise profile, but I'm getting 2 errors:
```
$ LC_ALL=C /home/paolo/git/darktable/tools/noise/darktable-gen-noiseprofile -d .
===> Check for required tools
--> Check for images handling tools availability
--> Check for images export tools availability
--> Check for profiling tools availability
--> Check for tethering tools availability
--> Check for pdf tools availability
===> Check profiling directory
===> List profiling input RAW images
--> Found ISO 125 image: ./G5Xm2_125.CR3
The script will use existing input RAW images for the profiling. No more
shot will be taken.
===> Checking profiling RAW images correctness + Jpeg export
--> ISO 125:
./G5Xm2_125.CR3
"./G5Xm2_125.CR3" not sufficiently over-exposed (0 / 80)
/home/paolo/git/darktable/tools/noise/subr.sh: line 496: [: 31658.5: integer expression expected
The following ISO settings have incorrect images:
o 125 ISO
Please read the error messages associated with each file above, check
your subject and re-do the shots.
```
The first error is that the [image](https://github.com/darktable-org/darktable/files/8241369/G5Xm2_125.zip) is correctly over-exposed, but the script keeps reporting that it's not
The second error is properly a bash bug: the "integer expression expected" output
Besides that, if more that one image is in the directory, and various produce error, only the last image producing error is reported when it says "The following ISO settings have incorrect images:"
I think the solution would be to turn of the “highlight reconstruction” module for JPEG conversion.
sugarbravo
(Šarūnas Burdulis)
April 6, 2022, 8:37pm
#175
Trying to follow:
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/opt/darktable -DBUILD_NOISE_TOOLS=ON ..
cd tools/noise
make
sudo make install
Change cd tools/noise
→ cd ../tools/noise
, but then:
$ make
make: *** No targets specified and no makefile found. Stop.
da-phil
April 16, 2022, 8:42pm
#176
What are you trying to achieve:
build the whole DT project with noise tools
build only noise tools?
in case of the latter, you can just:
cd tools/noise
mkdir build
cd build
cmake -DCMAKE_INSTALL_LIBEXECDIR=/opt/darktable ..
sudo make install
sugarbravo
(Šarūnas Burdulis)
April 22, 2022, 5:03pm
#177
I was trying to build darktable and noise tools.
So perhaps the instructions should be updated with separate build within tools/noise.
da-phil
April 22, 2022, 6:07pm
#178
Why did you do cd tools/noise
then?
Without that it should work and build the whole darktable project including the noise tools.
sugarbravo
(Šarūnas Burdulis)
April 22, 2022, 6:37pm
#179
Because darktable was built alright, but tools/noise were not.
da-phil
April 22, 2022, 8:24pm
#180
At least for me it works, just those steps (as you outlined already):
mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX=/opt/darktable -DBUILD_NOISE_TOOLS=ON ..
sudo make install
Somewhere during the build process you should come across this output:
Scanning dependencies of target darktable-noiseprofile
[ 9%] Building C object share/darktable/tools/noise/CMakeFiles/darktable-noiseprofile.dir/noiseprofile.c.o
After successfull build & installation you should find the noise calibration tools here: /opt/darktable/libexec/darktable/tools
sugarbravo
(Šarūnas Burdulis)
April 22, 2022, 10:39pm
#181
Thanks!
Just tried compiling entire darktable once more and I do get /opt/darktable/libexec/darktable/tools/darktable-noiseprofile
.
Then what is
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/opt/darktable -DBUILD_NOISE_TOOLS=ON ..
cd tools/noise
make
sudo make install
for? If compiling tools only, shouldn’t cd tools/noise
be the first line?
da-phil
April 23, 2022, 8:48am
#182
Where did you find this line cd tools/noise
in the instructions? It does not make any sense after you already changed into the build
folder.
If you only want to build the noise tools follow my instructions here: How to create camera noise profiles for darktable - #176 by da-phil