GMIC cli suddenly very slow

That’s correct indeed, mine is also very slow for this resolution.
I don’t know what happened, since I didn’t change the pixelsort command recently.
I’ll check that tonight if I have some time.
@IeBo8Aj , what was the resolution of your input image?

That may be a regression in latest version 3.1.3.

I’ve tested the filter with G’MIC 3.1.2, and it runs really fast compared to 3.1.3.
So clearly, there is a regression somewhere. I need to find what happened between the two versions and see if I can do something to improve this
(will require a new release, probably).

2 Likes

David, it looks like the resolution is:

Indeed, that is why I showed the timing on the previous version.

Some news: the regression in 3.1.3 is confirmed. It is actually comes from my attempt to optimize certain arithmetic operations, more precisely this (from the Changelog):

  • [core-313] Optimize syntax +command for some arithmetic commands (e.g. sqr, sqrt, cos, …).

Unfortunately, this indeed optimizes some operations, but it also slows down other a lot (it’s the case for +crop applied on a large image for instance, which is something done by command pixelsort.

Hopefully, I’ve been able to write a fix tonight.
I’ll release new version 3.1.4 tomorrow with this fix included.

Thanks @IeBo8Aj for your report, it helps building a better G’MIC ! :+1:


For more details, see fix here:

I’m wondering if that means you had to undo the optimisation, or if you got the best of both somehow?
Either way, I suppose we’ll find out with the next build…

No, I found a way to keep the optimization for the arithmetic functions, so at the end 3.1.4 will be indeed faster than 3.1.2 & 3.1.3

2 Likes

Nice! Yet another bonus then :slight_smile:

@David_Tschumperle Nice!
@IeBo8Aj Thanks for reporting.

1 Like

Version 3.1.4 of G’MIC has been released, with the fix included.

1 Like

Thank you @David_Tschumperle for identifying the optimisation problem and releasing a fix so quickly. Thanks too to @afre and @garagecoder for your troubleshooting suggestions.

FWIW, the execution time for my pixelsorting job with GMIC 3.1.3 is 198 seconds, compared with previous times <2 sec. Note that the source image is png format not jpg, in case this makes any difference.

[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input file 'input.png' at position 0 (1 image 3072x4080x1x3).
[gmic]-1./ Output image [0] as png file 'outfile.png' (1 image 3072x4080x1x3).
[gmic]-1./ End G'MIC interpreter.
gmic input.png fx_pixelsort 1,0,2,1,0,100,0,0,0 output outfile.png  198.80s user 89.66s system 99% cpu 4:49.03 total

I will run this job again after I have installed GMIC 3.1.4 … just waiting for it to hit the arch repositories.

Thanks again, everyone!

1 Like

Arch packagers are usually very reactive. This should not take too long :slight_smile:

1 Like

The arch repository has now been updated to include GMIC 3.1.4 and GMIC performance is back to normal. I was over-optimistic in my <2 sec execution estimate in the last post … around 5 sec would have been more accurate.

gmic infile.png fx_pixelsort 1,0,2,1,0,100,0,0,0 output outfile.png
6.56s user 0.16s system 105% cpu 6.344 total

This is so much better than yesterday’s 198 seconds!

Thanks again.

2 Likes

I actually think that the command pixelsort could be optimized a bit.
Not much time to look at this, but I keep this on my TODO.

1 Like

I reply to myself, but yes, I confirm something faster is possible.
I’ve a prototype of sorting that runs in parallel (e.g. each row sorted in parallel), and I get a x2 factor here. Will try to make this cleaner so I can replace the current pixelsort function by the parallelized version.

@IeBo8Aj , now that you have G’MIC 3.1.4, could you please try to update your commands ?

$ gmic up

I’ve pushed a modification in the pixelsort filter that enables parallelization and that should be faster for large images.

1 Like

Thanks, @David_Tschumperle. I’ve updated G’MIC 3.1.4 as per your instructions and run my script on a test image. It appears to be slightly slower for the same sized image (7.45 sec user versus 6.56 sec user), but I am not using exactly the same file that I used in previous tests, so it is not a properly controlled experiment. (I was using a test file which I have since deleted, thinking I would not be needing it anymore.) Time details below.

[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input file 'infile.png' at position 0 (1 image 3072x4080x1x3).
[gmic]-1./ Output image [0] as png file 'outfile.png' (1 image 3072x4080x1x3).
[gmic]-1./ End G'MIC interpreter.
gmic infile.png fx_pixelsort 1,0,2,1,0,100,0,0,0 output outfile.png  7.45s user 0.27s system 141% cpu 5.444 total

It would be helpful if we could test the same image. Please upload and/or send us a link.

Here’s a link to the input file. 19MB PNG.

With your file, I get this:

$ time gmic infile.png tic fx_pixelsort 1,0,2,1,0,100,0,0,0 toc output outfile.png
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input file 'infile.png' at position 0 (1 image 3072x4080x1x3).
[gmic]-1./ Initialize timer.
[gmic]-1./ Elapsed time: 1.075 s.
[gmic]-1./ Output image [0] as png file 'outfile.png' (1 image 3072x4080x1x3).
[gmic]-1./ End G'MIC interpreter.

real    0m4,494s
user    0m6,398s
sys     0m0,576s

Note that the most time is spent for saving the outfile.png file. The pixel sorting algorithm takes 1.075s here. I can’t do anything to speed up the saving, as it is something delegated to the libpng library.

I hope I didn’t come across as complaining about execution time with the new patch. I’m very happy with the performance, and understand that I/O will be the biggest botteneck.