GMIC cli suddenly very slow

Software G'MIC
1

My gmic scripts are running either very slowly, or not at all (just hanging). The scripts were running fine yesterday, but it’s a different story today. I suspect this morning’s system update may have broken something that gmic relies on, but I do not know what may have broken or how to fix it.

I’m running gmic 3.1.3 under Arch Linux.

My gmic jobs are quite simple e.g. Sorting pixels:

gmic infile.jpg fx_pixelsort 1,0,2,1,0,100,0,0,0 output outfile.jpg`
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input file 'input/junk.png' at position 0 (1 image 3072x4080x1x3).

This job just hangs appears to hang for several minutes before starting execution, but yesterday would run immediately and in a few seconds.

There is no shortage of system resources. PC has 16Gb RAM with only 3.2Gb used, so it’s not a shortage of memory. There’s plenty of disk space. Even after a reboot with no other Applications running, the script just crawls instead of running immediately and quickly. Something is not right.

I’m sorry that this is such a vague question … but currently the gmic cli is not usable for me and I don’t know what I can do to improve the situation. Does anyone have a similar experience?

2

Testing here with G’MIC 3.1.4_pre (basically same as 3.1.3) :

$ time gmic sp lena fx_pixelsort 1,0,2,1,0,100,0,0,0 output toto.png
[gmic]-0./ Start G'MIC interpreter.
[gmic]-1./ Input sample image 'lena' (1 image 512x512x1x3).
[gmic]-1./ Output image [0] as png file 'toto.png' (1 image 512x512x1x3).
[gmic]-1./ End G'MIC interpreter.

real	0m0,551s
user	0m0,983s
sys	0m0,030s

(Ubuntu 20.04, with latest update).

Maybe you could try adding the -debug option to see when and where it hangs ?

3

My Version 3.1.2 results in Elapsed time: 3.645 s.
I have an old damaged machine. :slight_smile:

4

I’m on an ubuntu derived OS with a reasonable laptop and mine also hangs for ages with this size of image. I’m curious how long it used to take, for the same resolution input?

5

That’s correct indeed, mine is also very slow for this resolution.
I don’t know what happened, since I didn’t change the pixelsort command recently.
I’ll check that tonight if I have some time.
@IeBo8Aj , what was the resolution of your input image?

That may be a regression in latest version 3.1.3.

6

I’ve tested the filter with G’MIC 3.1.2, and it runs really fast compared to 3.1.3.
So clearly, there is a regression somewhere. I need to find what happened between the two versions and see if I can do something to improve this
(will require a new release, probably).

2 Likes
7

David, it looks like the resolution is:

Indeed, that is why I showed the timing on the previous version.

8

Some news: the regression in 3.1.3 is confirmed. It is actually comes from my attempt to optimize certain arithmetic operations, more precisely this (from the Changelog):

  • [core-313] Optimize syntax +command for some arithmetic commands (e.g. sqr, sqrt, cos, …).

Unfortunately, this indeed optimizes some operations, but it also slows down other a lot (it’s the case for +crop applied on a large image for instance, which is something done by command pixelsort.

Hopefully, I’ve been able to write a fix tonight.
I’ll release new version 3.1.4 tomorrow with this fix included.

Thanks @IeBo8Aj for your report, it helps building a better G’MIC ! :+1:

For more details, see fix here:

9

I’m wondering if that means you had to undo the optimisation, or if you got the best of both somehow?
Either way, I suppose we’ll find out with the next build…

10

No, I found a way to keep the optimization for the arithmetic functions, so at the end 3.1.4 will be indeed faster than 3.1.2 & 3.1.3

2 Likes
11

Nice! Yet another bonus then :slight_smile:

12

@David_Tschumperle Nice!
@IeBo8Aj Thanks for reporting.

1 Like
13

Version 3.1.4 of G’MIC has been released, with the fix included.

1 Like
14

Thank you @David_Tschumperle for identifying the optimisation problem and releasing a fix so quickly. Thanks too to @afre and @garagecoder for your troubleshooting suggestions.

FWIW, the execution time for my pixelsorting job with GMIC 3.1.3 is 198 seconds, compared with previous times <2 sec. Note that the source image is png format not jpg, in case this makes any difference.

[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input file 'input.png' at position 0 (1 image 3072x4080x1x3).
[gmic]-1./ Output image [0] as png file 'outfile.png' (1 image 3072x4080x1x3).
[gmic]-1./ End G'MIC interpreter.
gmic input.png fx_pixelsort 1,0,2,1,0,100,0,0,0 output outfile.png  198.80s user 89.66s system 99% cpu 4:49.03 total

I will run this job again after I have installed GMIC 3.1.4 … just waiting for it to hit the arch repositories.

Thanks again, everyone!

1 Like
15

Arch packagers are usually very reactive. This should not take too long :slight_smile:

1 Like
16

The arch repository has now been updated to include GMIC 3.1.4 and GMIC performance is back to normal. I was over-optimistic in my <2 sec execution estimate in the last post … around 5 sec would have been more accurate.

gmic infile.png fx_pixelsort 1,0,2,1,0,100,0,0,0 output outfile.png
6.56s user 0.16s system 105% cpu 6.344 total

This is so much better than yesterday’s 198 seconds!

Thanks again.

2 Likes
17

I actually think that the command pixelsort could be optimized a bit.
Not much time to look at this, but I keep this on my TODO.

1 Like
18

I reply to myself, but yes, I confirm something faster is possible.
I’ve a prototype of sorting that runs in parallel (e.g. each row sorted in parallel), and I get a x2 factor here. Will try to make this cleaner so I can replace the current pixelsort function by the parallelized version.

19

@IeBo8Aj , now that you have G’MIC 3.1.4, could you please try to update your commands ?

$ gmic up

I’ve pushed a modification in the pixelsort filter that enables parallelization and that should be faster for large images.

1 Like
20

Thanks, @David_Tschumperle. I’ve updated G’MIC 3.1.4 as per your instructions and run my script on a test image. It appears to be slightly slower for the same sized image (7.45 sec user versus 6.56 sec user), but I am not using exactly the same file that I used in previous tests, so it is not a properly controlled experiment. (I was using a test file which I have since deleted, thinking I would not be needing it anymore.) Time details below.

[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input file 'infile.png' at position 0 (1 image 3072x4080x1x3).
[gmic]-1./ Output image [0] as png file 'outfile.png' (1 image 3072x4080x1x3).
[gmic]-1./ End G'MIC interpreter.
gmic infile.png fx_pixelsort 1,0,2,1,0,100,0,0,0 output outfile.png  7.45s user 0.27s system 141% cpu 5.444 total