Machine Learning Library in G'MIC

Succeeds here. I obtain five one column images:
cachedump

I obtained these images running:

gosgood@lydia ~ $ gmic debug input_cached gmic_denoise_cnn.gmz
[gmic]-0./ Start G'MIC interpreter (in debug mode).<gmic>-0./ Initial command line: 'cli_start , debug input_cached gmic_denoise_cnn.gmz'.
<gmic>./ Decompose command line into 5 items: 
<gmic>./   item[0] = 'cli_start'
<gmic>./   item[1] = ','
<gmic>./   item[2] = 'debug'
<gmic>./   item[3] = 'input_cached'
<gmic>./   item[4] = 'gmic_denoise_cnn.gmz'
…

I’m not going to post the entire debug dump, but, instead, include it in this dump file attachment.
inputcachedump.txt (65.2 KB)
I succeed in obtaining the denoising files; you fail. The question arises on how your debug dump file may differ; for those of us who have acquired the art of reading these dumps, we have a good chance of seeing the point of failure and then suggesting a course for you to proceed. So if you could, run the command as I have typed it, with the debug amendment, and post the results here. Thank you!

For completeness: after the various cp of the previous commands I removed the files added with rm and restored to the initial condition.

imputcachedumpfab7d.txt (54.8 KB) (attention I have updated the file one time)

…it talk about a magic number :thinking:

1 Like

Thank you for the dump file!

<gmic>-0./+input_cached/*if/*local/ Command 'input': arguments = '$url' -> 'https://gmic.eu/gmic_denoise_cnn.gmz'.
[gmic]-0./+input_cached/*if/*local/ Input file 'https://gmic.eu/gmic_denoise_cnn.gmz' at position 0
[gmic]-0./+input_cached/*if/*local/ *** Error *** Command 'input': File 'https://gmic.eu/gmic_denoise_cnn.gmz' is not in .gmz format (magic number not found).
[gmic]-0./+input_cached/*if/*local/ Reach 'onfail' block.

A ‘magic number’ is a short, distinctive number placed at the very start of the file to signify that it is of a definite type. It seems that gmic is downloading something, but the initial bytes of that something is not what a file of type .gmz would contain. Possibly communication corruption. Possibly a low-level connection reset, leading to an empty file — it has a file system name, but has zero bytes of content.

A successful run of input_cached places the obtained file in your cache. Following a successful run here, I have this listing in my `~/.cache/gmic file:

gosgood@bertha ~ $ ls -l ~/.cache/gmic
total 4064
-rw-r--r-- 1 gosgood gosgood 1111349 Apr 22 07:19 gmic_denoise_cnn.gmz     <===
-rw-r--r-- 1 gosgood gosgood  799101 Feb  3 06:53 sample_car.png
-rw-r--r-- 1 gosgood gosgood  524736 Nov 24 09:39 sample_cat.png
-rw-r--r-- 1 gosgood gosgood  438447 Nov 27 12:14 sample_lena.png
-rw-r--r-- 1 gosgood gosgood  609594 Feb  3 06:53 sample_monkey.png
-rw-r--r-- 1 gosgood gosgood  273917 Nov  8 07:38 sample_rose.png
-rw-r--r-- 1 gosgood gosgood  387660 Apr 17 06:46 sample_square.png

On your machine, is the file (a) missing? or (b) present, but size zero, or (c) present, but some size other than what I have?

If you have linux, some external tools may be of some use. First, octal dump od. Here is an alphanumeric dump of the beginning of gmic_denoise_cnn.gmz

$ od -a ~/.cache/gmic/gmic_denoise_cnn.gmz | less

0000000   6  sp   u   i   n   t   8  sp   l   i   t   t   l   e   _   e
0000020   n   d   i   a   n  nl   1  sp   3   6   1   7   9   9  sp   1
0000040  sp   1  sp   #   3   6   1   9   2   0  nl   x  fs nul  vt   @
…

If (c), present, but some size, other than 1111349 bytes, what does your dump look like?

Finally, does wget obtain the file without problems (unlike gmic), or does it too fail for some reason? Here, I can fetch the .gmz file without issue:

gosgood@bertha ~ $ wget https://gmic.eu/gmic_denoise_cnn.gmz
--2023-04-22 07:08:54--  https://gmic.eu/gmic_denoise_cnn.gmz
Resolving gmic.eu... 213.186.33.40
Connecting to gmic.eu|213.186.33.40|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1111349 (1.1M)
Saving to: ‘gmic_denoise_cnn.gmz’

gmic_denoise_cnn.gm 100%[===================>]   1.06M   182KB/s    in 6.0s    

2023-04-22 07:09:00 (180 KB/s) - ‘gmic_denoise_cnn.gmz’ saved [1111349/1111349]

What happens in your case? If a fetch fails with wget, then there is some issue outside of gmic with which to contend…

Thanks!

ls -l ~/.cache/gmic
totale 10124
-rw-rw-r-- 1 fabio fabio   13235 17 apr 17.05 clut_fuji_xtrans_iii_provia.cimgz
-rw-rw-r-- 1 fabio fabio   12977 17 apr 17.05 clut_fuji_xtrans_iii_velvia.cimgz
-rw-rw-r-- 1 fabio fabio  123441 17 apr 17.05 clut_kodak_elite_extracolor_100.cimgz
-rw-rw-r-- 1 fabio fabio   12901 17 apr 17.05 clut_polaroid_px-680_-.cimgz
-rw-rw-r-- 1 fabio fabio 1111349 21 apr 21.05 gmic_denoise_cnn.gmz
-rw-rw-r-- 1 fabio fabio   48175 18 lug  2022 grain_orwo_np20.cimgz
-rw-rw-r-- 1 fabio fabio  791687  1 ago  2022 lightleak_000000.cimgz
-rw-rw-r-- 1 fabio fabio  220627 21 apr 19.49 sample_bottles.png
-rw-rw-r-- 1 fabio fabio  961577 21 apr 21.13 sample_dog.png
-rw-rw-r-- 1 fabio fabio  376133 21 apr 19.47 sample_eagle.png
-rw-rw-r-- 1 fabio fabio  379541 21 apr 21.23 sample_earth.png
-rw-rw-r-- 1 fabio fabio  282923 21 apr 21.04 sample_fruits.png
-rw-rw-r-- 1 fabio fabio  242811 21 apr 21.07 sample_gmicky.png
-rw-rw-r-- 1 fabio fabio  884898 21 apr 19.50 sample_landscape.png
-rw-rw-r-- 1 fabio fabio  385078 21 apr 21.23 sample_leaf.png
-rw-rw-r-- 1 fabio fabio  438447 21 apr 21.09 sample_lena.png
-rw-rw-r-- 1 fabio fabio  427358 21 apr 19.49 sample_leno.png
-rw-rw-r-- 1 fabio fabio  594816 21 apr 21.07 sample_mandrill.png
-rw-rw-r-- 1 fabio fabio  654655 21 apr 21.23 sample_portrait5.png
-rw-rw-r-- 1 fabio fabio  787955 21 apr 21.11 sample_portrait8.png
-rw-rw-r-- 1 fabio fabio  747609 21 apr 21.08 sample_rooster.png
-rw-rw-r-- 1 fabio fabio  812578 21 apr 21.33 sample_swan.png

$ wget https://gmic.eu/gmic_denoise_cnn.gmz
--2023-04-22 14:23:43--  https://gmic.eu/gmic_denoise_cnn.gmz
Risoluzione di gmic.eu... 213.186.33.40
Connessione a gmic.eu|213.186.33.40|:443... connesso.
Richiesta HTTP inviata, in attesa di risposta... 200 OK
Lunghezza: 1111349 (1,1M)
Salvataggio in: «gmic_denoise_cnn.gmz»

gmic_denoise_cnn.gmz              100%[===========================================================>]   1,06M  2,88MB/s    in 0,4s    

2023-04-22 14:23:44 (2,88 MB/s) - «gmic_denoise_cnn.gmz» salvato [1111349/1111349]

 $ od -a ~/.cache/gmic/gmic_denoise_cnn.gmz | less

0000000   6  sp   u   i   n   t   8  sp   l   i   t   t   l   e   _   e
0000020   n   d   i   a   n  nl   1  sp   3   6   1   7   9   9  sp   1
0000040  sp   1  sp   #   3   6   1   9   2   0  nl   x  fs nul  vt   @
0000060   t   ?   1   3  sp   f   l   o   a   t   3   2  sp   l   i   t
0000100   t   l   e   _   e   n   d   i   a   n  nl   2   8  sp   6   4
....

The file gmic_denoise_cnn.gmz was not in ./cache/gmic I downloaded it following the advice of David_Tschumperle:

@FAb7D

To try to help here are some verification tests to do in root mode and give the results.
Successively:

cd / 
find -iname "gmic_denoise_cnn.gmz"
find -iname "gmic"
find -iname "gmic_qt"
find -iname "gmic_gimp_qt"
$ sudo find -iname "gmic_denoise_cnn.gmz"
./home/fabio/.config/GIMP/2.10/plug-ins/gmic_gimp_qt/gmic_denoise_cnn.gmz
./home/fabio/.config/GIMP/2.10/plug-ins/gmic_denoise_cnn.gmz
./home/fabio/Scaricati/gmic_denoise_cnn.gmz
./home/fabio/.cache/gmic/gmic_denoise_cnn.gmz


 $ sudo find -iname "gmic"
./home/fabio/.cache/gmic
./usr/share/gmic
./usr/share/bash-completion/completions/gmic
./usr/bin/gmic

 $ sudo find -iname "gmic_qt"
./usr/bin/gmic_qt

 $ sudo find -iname "gmic_gimp_qt"
./home/fabio/.config/GIMP/2.10/plug-ins/gmic_gimp_qt
./usr/lib64/gimp/2.0/plug-ins/gmic_gimp_qt
./usr/lib64/gimp/2.0/plug-ins/gmic_gimp_qt/gmic_gimp_qt

Also this two are my work (added “manualy”):
./home/fabio/.config/GIMP/2.10/plug-ins/gmic_gimp_qt/gmic_denoise_cnn.gmz
./home/fabio/.config/GIMP/2.10/plug-ins/gmic_denoise_cnn.gmz

@FAb7D

1 - Can you try to delete all files ‘gmic_denoise_cnn.gmz’

2 - Correct by deleting the file or directory

./home/fabio/.config/GIMP/2.10/plug-ins/gmic_gimp_qt

The rest of the files are good.

3 - Retry the command in normal and root mode:

gmic sp dog noise 30 +denoise_cnn 0

4 - Then in root mode:

cd / 
find -iname "gmic_denoise_cnn.gmz"
 $ rm .config/GIMP/2.10/plug-ins/gmic_gimp_qt/gmic_denoise_cnn.gmz
 $ rm .config/GIMP/2.10/plug-ins/gmic_denoise_cnn.gmz
 $ rm .cache/gmic/gmic_denoise_cnn.gmz
$ rm -r .config/GIMP/2.10/plug-ins/gmic_gimp_qt
$ rm ./Scaricati/gmic_denoise_cnn.gmz
$ gmic sp dog noise 30 +denoise_cnn 0
[gmic]-0./ Start G'MIC interpreter.
[gmic]-1./ Input sample image 'dog' (1 image 1024x685x1x3).
[gmic]-1./ Add gaussian noise to image [0], with standard deviation 30.
[gmic]-1./ Denoise image [0] using a convolutional neural network (for soft-noise), with patch size 64.
[gmic] *** Error in ./ *** Command 'denoise_cnn': Unable to load network file 'gmic_denoise_cnn.gmz'.

$ sudo gmic sp dog noise 30 +denoise_cnn 0
[sudo] password for fabio:
[gmic]-0./ Start G'MIC interpreter.
[gmic]-1./ Input sample image 'dog' (1 image 1024x685x1x3).
[gmic]-1./ Add gaussian noise to image [0], with standard deviation 30.
[gmic]-1./ Denoise image [0] using a convolutional neural network (for soft-noise), with patch size 64.
[gmic] *** Error in ./ *** Command 'denoise_cnn': Unable to load network file 'gmic_denoise_cnn.gmz'.

 $ cd /
$ sudo find -iname "gmic_denoise_cnn.gmz"
$




@FAb7D

There may have been a bad configuration or installation at some point.
It is possible to resume a complete installation.
In this case you must save the filters you have created as well as your filter settings.
Use the packet manager to uninstall G’MIC.
Delete in root mode all that remains of G’MIC CLI, QT, GIMP (files and directories) and which does not need to be saved.
Download the latest ‘Full’ version of G’MIC.
With me it’s ‘GMIC Ubuntu 22.04 Jammy: Full Package’ , then i install the package ‘GMIC_3.2.3_UBuntu22-04_jammy_amd64.deb’.
If the problem persists it will not be linked to G’MIC.
:o)

…bad news… anyway thanks :roll_eyes:

@FAb7D
You may be able to create a new user and try.

@FAb7D , could you try this :

$ gmic debug input_cached gmic_denoise_cnn.gmz > log.txt

and copy/paste the content of the log.txt file?
It would help to understand what is wrong.
Thanks!

Of course
log.txt (64.8 KB)
Thanks!

I asked in the Solus forum if anyone has the same problem and it seems so.
https://discuss.getsol.us/d/9182-gmic-repair-denoise-dont-work

Thanks, and could you tell me in which folder the file gmic_denoise_cnn.gmz was located when you run the test?

In this moment there isn’t becouse I removed for this

but if I add on ~/.cache/gmic with:

~/.cache/gmic $ wget https://gmic.eu/gmic_denoise_cnn.gmz

the new

~ $ gmic debug input_cached gmic_denoise_cnn.gmz > log2.txt

become this:
log2.txt (64.9 KB)
(so in this case gmic_denoise_cnn.gmz is located on ~/.cache/gmic)

1 Like

and…I’m sorry… There is already a version prior to the changes made with SAMJ…
Is here
https://discuss.pixls.us/t/machine-learning-library-in-gmic/25746/54?u=fab7d
In subsequent posts there are also other tests… it doesn’t recognize this magic number, everything points to a problem in the package.

Ok, so we are going forward.
Now, the thing we have to make work is simply :

$ gmic gmic_denoise_cnn.gmz

where you run gmic in the folder where gmic_denoise_cnn.gmz is located.
Does this work for you (it should not).

Could you tell me what version of G’MIC you use ?

Also, what gives:

$ gmic it gmic_denoise_cnn.gmz unserialize

?

Here you are!

~/.cache/gmic $ ls gm*
gmic_denoise_cnn.gmz

~/.cache/gmic $ gmic gmic_denoise_cnn.gmz
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input file 'gmic_denoise_cnn.gmz' at position 0
[gmic]-0./ *** Error *** Command 'input': File 'gmic_denoise_cnn.gmz' is not in .gmz format (magic number not found).

 ~/.cache/gmic $ gmic it gmic_denoise_cnn.gmz unserialize
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input text-data file 'gmic_denoise_cnn.gmz'.
[gmic]-1./ Unserialize image [0].
[gmic]-1./ *** Error *** Command 'unserialize': gmicList<float>::unserialize(): Invalid specified size (0,0,0,0) for image #1 in serialized buffer.
[gmic] Command 'unserialize' has the following description: 

  unserialize (+):

    Recreate lists of images from serialized image buffers, obtained with 
    command 'serialize'.

 ~/.cache/gmic $ gmic
  gmic: GREYC's Magic for Image Computing: command-line interface
        Version 3.0.2
       ...

OK, so I think I’ve found what happens.
You are using version 3.0.2 of G’MIC, and there was a name change for value type in the .gmz and .cimgz that has been introduced in G’MIC 3.1.0.
Excerpt from the changelog:

* *[core-310]* Names of pixel types have been normalized. Now, they can be `{ bool | uint8 | int8 | uint16 | int16 | uint32 | int32 | uint64 | int64 | float32 | float64 }`.

In particular, pixel type uint8 replaces old uchar, and float32 replaces old float.

These new names are used in the header of the file gmic_denoise_cnn.gmz because this file has been updated since 3.1.0, and while newer versions of G’MIC are able to read .gmz files with the older type names, the contrary is not true.

I’m then a bit sorry to say this, but you won’t be able to run command denoise_cnn with G’MIC 3.0.2.

Could it be a good occasion for the packagers of Solus to update G’MIC to the latest stable version 3.2.3 ?

2 Likes