Hello David,
I don’t know if it’s helpful: There is an interesting project at [GitHub - Vandermode/ELD: A Physics-based Noise Formation Model for Extreme Low-light Raw Denoising (CVPR 2020 Oral & TPAMI 2021)], but unfortunately it doesn’t contain a user interface.
But it shares the dataset [ELD] in GoogleDrive:
[ELD – Google Drive]
These are high resolution photos of different camera models in RAW format.
With best regards
Jörn
Hello, I’m trying denoise but I get the following error:
$ gmic sp prova-1.jpg , 256 noise 15 cut 0,255 +denoise_cnn 0
[gmic]-0./ Start G’MIC interpreter.
[gmic]-1./ Input sample image ‘landscape’ (1 image 1024x576x1x3).
[gmic]-1./ Input file ‘prova-1.jpg’ at position 1 (1 image 6024x4022x1x3).
[gmic]-2./ Input black image at position 2 (1 image 256x1x1x1).
[gmic]-3./ Add gaussian noise to images [0,1,2], with standard deviation 15.
[gmic]-3./ Cut images [0,1,2] in range [0,255].
[gmic]-3./ Denoise images [0,1,2] using a convolutional neural network (for soft-noise), with patch size 64.
[gmic] *** Error in ./ *** Command ‘denoise_cnn’: Unable to load network file ‘gmic_denoise_cnn.gmz’.
Is it a Solus (my distro) package problem or something else?
This command requires the neural network weights to run properly.
They can be downloaded from here : https://gmic.eu/gmic_denoise_cnn.gmz
(they should actually be downloaded automatically by the denoise_cnn command, but if you have some firewalls or security rules, G’MIC won’t be able to download this file).
Workaround : download this file and put it in $HOME/.cache/gmic, the command should work then.
It still doesn’t work:
~/.cache/gmic $ ls -l
totale 9328
-rw-rw-r-- 1 fabio fabio 13235 17 apr 17.05 clut_fuji_xtrans_iii_provia.cimgz
-rw-rw-r-- 1 fabio fabio 12977 17 apr 17.05 clut_fuji_xtrans_iii_velvia.cimgz
-rw-rw-r-- 1 fabio fabio 123441 17 apr 17.05 clut_kodak_elite_extracolor_100.cimgz
-rw-rw-r-- 1 fabio fabio 12901 17 apr 17.05 clut_polaroid_px-680_-.cimgz
-rwxrwxr-x 1 fabio fabio 1111349 21 apr 21.05 gmic_denoise_cnn.gmz
-rw-rw-r-- 1 fabio fabio 48175 18 lug 2022 grain_orwo_np20.cimgz
-rw-rw-r-- 1 fabio fabio 791687 1 ago 2022 lightleak_000000.cimgz
-rw-rw-r-- 1 fabio fabio 220627 21 apr 19.49 sample_bottles.png
-rw-rw-r-- 1 fabio fabio 961577 21 apr 21.13 sample_dog.png
-rw-rw-r-- 1 fabio fabio 376133 21 apr 19.47 sample_eagle.png
-rw-rw-r-- 1 fabio fabio 379541 21 apr 21.23 sample_earth.png
-rw-rw-r-- 1 fabio fabio 282923 21 apr 21.04 sample_fruits.png
-rw-rw-r-- 1 fabio fabio 242811 21 apr 21.07 sample_gmicky.png
-rw-rw-r-- 1 fabio fabio 884898 21 apr 19.50 sample_landscape.png
-rw-rw-r-- 1 fabio fabio 385078 21 apr 21.23 sample_leaf.png
-rw-rw-r-- 1 fabio fabio 438447 21 apr 21.09 sample_lena.png
-rw-rw-r-- 1 fabio fabio 427358 21 apr 19.49 sample_leno.png
-rw-rw-r-- 1 fabio fabio 594816 21 apr 21.07 sample_mandrill.png
-rw-rw-r-- 1 fabio fabio 654655 21 apr 21.23 sample_portrait5.png
-rw-rw-r-- 1 fabio fabio 787955 21 apr 21.11 sample_portrait8.png
-rw-rw-r-- 1 fabio fabio 747609 21 apr 21.08 sample_rooster.png
Now I have many sample.png 
but:
$ gmic sp prova-1.jpg , 256 noise 15 cut 0,255 +denoise_cnn 0
[gmic]-0./ Start G'MIC interpreter.
[gmic]-1./ Input sample image 'portrait5' (1 image 800x800x1x3).
[gmic]-1./ Input file 'prova-1.jpg' at position 1 (1 image 6024x4022x1x3).
[gmic]-2./ Input black image at position 2 (1 image 256x1x1x1).
[gmic]-3./ Add gaussian noise to images [0,1,2], with standard deviation 15.
[gmic]-3./ Cut images [0,1,2] in range [0,255].
[gmic]-3./ Denoise images [0,1,2] using a convolutional neural network (for soft-noise), with patch size 64.
[gmic] *** Error in ./ *** Command 'denoise_cnn': Unable to load network file 'gmic_denoise_cnn.gmz'.
$ gmic sp dog noise 30 +denoise_cnn 0
[gmic]-0./ Start G'MIC interpreter.
[gmic]-1./ Input sample image 'dog' (1 image 1024x685x1x3).
[gmic]-1./ Add gaussian noise to image [0], with standard deviation 30.
[gmic]-1./ Denoise image [0] using a convolutional neural network (for soft-noise), with patch size 64.
[gmic] *** Error in ./ *** Command 'denoise_cnn': Unable to load network file 'gmic_denoise_cnn.gmz'.
The same, with and without firewall and VPN on.
~ $ gmic sp dog noise 30 +denoise_cnn 0
[gmic]-0./ Start G'MIC interpreter.
[gmic]-1./ Input sample image 'dog' (1 image 1024x685x1x3).
[gmic]-1./ Add gaussian noise to image [0], with standard deviation 30.
[gmic]-1./ Denoise image [0] using a convolutional neural network (for soft-noise), with patch size 64.
[gmic] *** Error in ./ *** Command 'denoise_cnn': Unable to load network file 'gmic_denoise_cnn.gmz'.
Well, we have to understand what is going on.
Basically, the command that loads the G’MIC resources datafiles is input_cached.
So if we succeed in doing this, without errors:
$ gmic input_cached gmic_denoise_cnn.gmz
Then the command denoise_cnn will work.
The command input_cached tries to locate the specified filename in these folders:
/usr/lib/gimp/2.0/plug-ins/path_cache, which is something you can see here:$ gmic e \$\{-path_cache\}
[gmic]-0./ Start G'MIC interpreter.
/home/tschumperle/.cache/gmic/
[gmic]-0./ End G'MIC interpreter.
So, basically, it’s the folder $HOME/.cache/gmic/.
path_gimp/plug-ins/ where path_gimp is given by:$ gmic e \$\{-path_gimp\}
[gmic]-0./ Start G'MIC interpreter.
/home/tschumperle/.config/GIMP/2.10/
[gmic]-0./ End G'MIC interpreter.
So for me, it’s: $HOME/config/GIMP/2.10/plug-ins/.
path_gimp/plug-ins/gmic_gimp_qt/
/usr/share/gmic/
So, basically if you manually download the file gmic_denoise_cnn.gmz and you put it in one of these folders, the input_cached command should succeed.
Can you do the test and tell us what happens?
I don’t know if I understood correctly, but I did some tests.
$ gmic e \$\{-path_cache\}
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ /home/fabio/.cache/gmic/
[gmic]-0./ End G'MIC interpreter.
$ gmic e \$\{-path_gimp\}
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ /home/fabio/.config/GIMP/2.10/
[gmic]-0./ End G'MIC interpreter.
$ gmic input_cached gmic_denoise_cnn.gmz
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input cached file 'gmic_denoise_cnn.gmz'.
[gmic] *** Error in ./ *** Command 'input_cached': Unknown filename 'gmic_denoise_cnn.gmz'.
$ cp /home/fabio/.cache/gmic/gmic_denoise_cnn.gmz /home/fabio/.config/GIMP/2.10/plug-ins/
$ gmic input_cached gmic_denoise_cnn.gmz
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input cached file 'gmic_denoise_cnn.gmz'.
[gmic] *** Error in ./ *** Command 'input_cached': Unknown filename 'gmic_denoise_cnn.gmz'.
sudo cp /home/fabio/.cache/gmic/gmic_denoise_cnn.gmz /usr/lib/gimp/2.0/plug-ins/
$ gmic input_cached gmic_denoise_cnn.gmz
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input cached file 'gmic_denoise_cnn.gmz'.
[gmic] *** Error in ./ *** Command 'input_cached': Unknown filename 'gmic_denoise_cnn.gmz'.
sudo cp /home/fabio/.cache/gmic/gmic_denoise_cnn.gmz /usr/share/gmic
$ gmic input_cached gmic_denoise_cnn.gmz
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input cached file 'gmic_denoise_cnn.gmz'.
[gmic] *** Error in ./ *** Command 'input_cached': Unknown filename 'gmic_denoise_cnn.gmz'.
$ls /home/fabio/.config/GIMP/2.10/plug-ins/q*
ls: impossibile accedere a '/home/fabio/.config/GIMP/2.10/plug-ins/q*': File o directory non esistente
$ locate gmic_gimp_qt
/usr/lib64/gimp/2.0/plug-ins/gmic_gimp_qt
/usr/lib64/gimp/2.0/plug-ins/gmic_gimp_qt/gmic_gimp_qt
I hope I did right
Thanks anyway!
Succeeds here. I obtain five one column images:
I obtained these images running:
gosgood@lydia ~ $ gmic debug input_cached gmic_denoise_cnn.gmz
[gmic]-0./ Start G'MIC interpreter (in debug mode).<gmic>-0./ Initial command line: 'cli_start , debug input_cached gmic_denoise_cnn.gmz'.
<gmic>./ Decompose command line into 5 items:
<gmic>./ item[0] = 'cli_start'
<gmic>./ item[1] = ','
<gmic>./ item[2] = 'debug'
<gmic>./ item[3] = 'input_cached'
<gmic>./ item[4] = 'gmic_denoise_cnn.gmz'
…
I’m not going to post the entire debug dump, but, instead, include it in this dump file attachment.
inputcachedump.txt (65.2 KB)
I succeed in obtaining the denoising files; you fail. The question arises on how your debug dump file may differ; for those of us who have acquired the art of reading these dumps, we have a good chance of seeing the point of failure and then suggesting a course for you to proceed. So if you could, run the command as I have typed it, with the debug amendment, and post the results here. Thank you!
For completeness: after the various cp of the previous commands I removed the files added with rm and restored to the initial condition.
imputcachedumpfab7d.txt (54.8 KB) (attention I have updated the file one time)
…it talk about a magic number …
Thank you for the dump file!
<gmic>-0./+input_cached/*if/*local/ Command 'input': arguments = '$url' -> 'https://gmic.eu/gmic_denoise_cnn.gmz'.
[gmic]-0./+input_cached/*if/*local/ Input file 'https://gmic.eu/gmic_denoise_cnn.gmz' at position 0
[gmic]-0./+input_cached/*if/*local/ *** Error *** Command 'input': File 'https://gmic.eu/gmic_denoise_cnn.gmz' is not in .gmz format (magic number not found).
[gmic]-0./+input_cached/*if/*local/ Reach 'onfail' block.
A ‘magic number’ is a short, distinctive number placed at the very start of the file to signify that it is of a definite type. It seems that gmic is downloading something, but the initial bytes of that something is not what a file of type .gmz would contain. Possibly communication corruption. Possibly a low-level connection reset, leading to an empty file — it has a file system name, but has zero bytes of content.
A successful run of input_cached places the obtained file in your cache. Following a successful run here, I have this listing in my `~/.cache/gmic file:
gosgood@bertha ~ $ ls -l ~/.cache/gmic
total 4064
-rw-r--r-- 1 gosgood gosgood 1111349 Apr 22 07:19 gmic_denoise_cnn.gmz <===
-rw-r--r-- 1 gosgood gosgood 799101 Feb 3 06:53 sample_car.png
-rw-r--r-- 1 gosgood gosgood 524736 Nov 24 09:39 sample_cat.png
-rw-r--r-- 1 gosgood gosgood 438447 Nov 27 12:14 sample_lena.png
-rw-r--r-- 1 gosgood gosgood 609594 Feb 3 06:53 sample_monkey.png
-rw-r--r-- 1 gosgood gosgood 273917 Nov 8 07:38 sample_rose.png
-rw-r--r-- 1 gosgood gosgood 387660 Apr 17 06:46 sample_square.png
On your machine, is the file (a) missing? or (b) present, but size zero, or (c) present, but some size other than what I have?
If you have linux, some external tools may be of some use. First, octal dump od. Here is an alphanumeric dump of the beginning of gmic_denoise_cnn.gmz
$ od -a ~/.cache/gmic/gmic_denoise_cnn.gmz | less
0000000 6 sp u i n t 8 sp l i t t l e _ e
0000020 n d i a n nl 1 sp 3 6 1 7 9 9 sp 1
0000040 sp 1 sp # 3 6 1 9 2 0 nl x fs nul vt @
…
If (c), present, but some size, other than 1111349 bytes, what does your dump look like?
Finally, does wget obtain the file without problems (unlike gmic), or does it too fail for some reason? Here, I can fetch the .gmz file without issue:
gosgood@bertha ~ $ wget https://gmic.eu/gmic_denoise_cnn.gmz
--2023-04-22 07:08:54-- https://gmic.eu/gmic_denoise_cnn.gmz
Resolving gmic.eu... 213.186.33.40
Connecting to gmic.eu|213.186.33.40|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1111349 (1.1M)
Saving to: ‘gmic_denoise_cnn.gmz’
gmic_denoise_cnn.gm 100%[===================>] 1.06M 182KB/s in 6.0s
2023-04-22 07:09:00 (180 KB/s) - ‘gmic_denoise_cnn.gmz’ saved [1111349/1111349]
What happens in your case? If a fetch fails with wget, then there is some issue outside of gmic with which to contend…
Thanks!
ls -l ~/.cache/gmic
totale 10124
-rw-rw-r-- 1 fabio fabio 13235 17 apr 17.05 clut_fuji_xtrans_iii_provia.cimgz
-rw-rw-r-- 1 fabio fabio 12977 17 apr 17.05 clut_fuji_xtrans_iii_velvia.cimgz
-rw-rw-r-- 1 fabio fabio 123441 17 apr 17.05 clut_kodak_elite_extracolor_100.cimgz
-rw-rw-r-- 1 fabio fabio 12901 17 apr 17.05 clut_polaroid_px-680_-.cimgz
-rw-rw-r-- 1 fabio fabio 1111349 21 apr 21.05 gmic_denoise_cnn.gmz
-rw-rw-r-- 1 fabio fabio 48175 18 lug 2022 grain_orwo_np20.cimgz
-rw-rw-r-- 1 fabio fabio 791687 1 ago 2022 lightleak_000000.cimgz
-rw-rw-r-- 1 fabio fabio 220627 21 apr 19.49 sample_bottles.png
-rw-rw-r-- 1 fabio fabio 961577 21 apr 21.13 sample_dog.png
-rw-rw-r-- 1 fabio fabio 376133 21 apr 19.47 sample_eagle.png
-rw-rw-r-- 1 fabio fabio 379541 21 apr 21.23 sample_earth.png
-rw-rw-r-- 1 fabio fabio 282923 21 apr 21.04 sample_fruits.png
-rw-rw-r-- 1 fabio fabio 242811 21 apr 21.07 sample_gmicky.png
-rw-rw-r-- 1 fabio fabio 884898 21 apr 19.50 sample_landscape.png
-rw-rw-r-- 1 fabio fabio 385078 21 apr 21.23 sample_leaf.png
-rw-rw-r-- 1 fabio fabio 438447 21 apr 21.09 sample_lena.png
-rw-rw-r-- 1 fabio fabio 427358 21 apr 19.49 sample_leno.png
-rw-rw-r-- 1 fabio fabio 594816 21 apr 21.07 sample_mandrill.png
-rw-rw-r-- 1 fabio fabio 654655 21 apr 21.23 sample_portrait5.png
-rw-rw-r-- 1 fabio fabio 787955 21 apr 21.11 sample_portrait8.png
-rw-rw-r-- 1 fabio fabio 747609 21 apr 21.08 sample_rooster.png
-rw-rw-r-- 1 fabio fabio 812578 21 apr 21.33 sample_swan.png
$ wget https://gmic.eu/gmic_denoise_cnn.gmz
--2023-04-22 14:23:43-- https://gmic.eu/gmic_denoise_cnn.gmz
Risoluzione di gmic.eu... 213.186.33.40
Connessione a gmic.eu|213.186.33.40|:443... connesso.
Richiesta HTTP inviata, in attesa di risposta... 200 OK
Lunghezza: 1111349 (1,1M)
Salvataggio in: «gmic_denoise_cnn.gmz»
gmic_denoise_cnn.gmz 100%[===========================================================>] 1,06M 2,88MB/s in 0,4s
2023-04-22 14:23:44 (2,88 MB/s) - «gmic_denoise_cnn.gmz» salvato [1111349/1111349]
$ od -a ~/.cache/gmic/gmic_denoise_cnn.gmz | less
0000000 6 sp u i n t 8 sp l i t t l e _ e
0000020 n d i a n nl 1 sp 3 6 1 7 9 9 sp 1
0000040 sp 1 sp # 3 6 1 9 2 0 nl x fs nul vt @
0000060 t ? 1 3 sp f l o a t 3 2 sp l i t
0000100 t l e _ e n d i a n nl 2 8 sp 6 4
....
The file gmic_denoise_cnn.gmz was not in ./cache/gmic I downloaded it following the advice of David_Tschumperle:
To try to help here are some verification tests to do in root mode and give the results.
Successively:
cd /
find -iname "gmic_denoise_cnn.gmz"
find -iname "gmic"
find -iname "gmic_qt"
find -iname "gmic_gimp_qt"
$ sudo find -iname "gmic_denoise_cnn.gmz"
./home/fabio/.config/GIMP/2.10/plug-ins/gmic_gimp_qt/gmic_denoise_cnn.gmz
./home/fabio/.config/GIMP/2.10/plug-ins/gmic_denoise_cnn.gmz
./home/fabio/Scaricati/gmic_denoise_cnn.gmz
./home/fabio/.cache/gmic/gmic_denoise_cnn.gmz
$ sudo find -iname "gmic"
./home/fabio/.cache/gmic
./usr/share/gmic
./usr/share/bash-completion/completions/gmic
./usr/bin/gmic
$ sudo find -iname "gmic_qt"
./usr/bin/gmic_qt
$ sudo find -iname "gmic_gimp_qt"
./home/fabio/.config/GIMP/2.10/plug-ins/gmic_gimp_qt
./usr/lib64/gimp/2.0/plug-ins/gmic_gimp_qt
./usr/lib64/gimp/2.0/plug-ins/gmic_gimp_qt/gmic_gimp_qt
Also this two are my work (added “manualy”):
./home/fabio/.config/GIMP/2.10/plug-ins/gmic_gimp_qt/gmic_denoise_cnn.gmz
./home/fabio/.config/GIMP/2.10/plug-ins/gmic_denoise_cnn.gmz
1 - Can you try to delete all files ‘gmic_denoise_cnn.gmz’
2 - Correct by deleting the file or directory
./home/fabio/.config/GIMP/2.10/plug-ins/gmic_gimp_qt
The rest of the files are good.
3 - Retry the command in normal and root mode:
gmic sp dog noise 30 +denoise_cnn 0
4 - Then in root mode:
cd /
find -iname "gmic_denoise_cnn.gmz"
$ rm .config/GIMP/2.10/plug-ins/gmic_gimp_qt/gmic_denoise_cnn.gmz
$ rm .config/GIMP/2.10/plug-ins/gmic_denoise_cnn.gmz
$ rm .cache/gmic/gmic_denoise_cnn.gmz
$ rm -r .config/GIMP/2.10/plug-ins/gmic_gimp_qt
$ rm ./Scaricati/gmic_denoise_cnn.gmz
$ gmic sp dog noise 30 +denoise_cnn 0
[gmic]-0./ Start G'MIC interpreter.
[gmic]-1./ Input sample image 'dog' (1 image 1024x685x1x3).
[gmic]-1./ Add gaussian noise to image [0], with standard deviation 30.
[gmic]-1./ Denoise image [0] using a convolutional neural network (for soft-noise), with patch size 64.
[gmic] *** Error in ./ *** Command 'denoise_cnn': Unable to load network file 'gmic_denoise_cnn.gmz'.
$ sudo gmic sp dog noise 30 +denoise_cnn 0
[sudo] password for fabio:
[gmic]-0./ Start G'MIC interpreter.
[gmic]-1./ Input sample image 'dog' (1 image 1024x685x1x3).
[gmic]-1./ Add gaussian noise to image [0], with standard deviation 30.
[gmic]-1./ Denoise image [0] using a convolutional neural network (for soft-noise), with patch size 64.
[gmic] *** Error in ./ *** Command 'denoise_cnn': Unable to load network file 'gmic_denoise_cnn.gmz'.
$ cd /
$ sudo find -iname "gmic_denoise_cnn.gmz"
$