I’ve tested the filter with G’MIC 3.1.2, and it runs really fast compared to 3.1.3.
So clearly, there is a regression somewhere. I need to find what happened between the two versions and see if I can do something to improve this
(will require a new release, probably).
David, it looks like the resolution is:
Indeed, that is why I showed the timing on the previous version.
Some news: the regression in 3.1.3 is confirmed. It is actually comes from my attempt to optimize certain arithmetic operations, more precisely this (from the Changelog):
-
[core-313] Optimize syntax
+command
for some arithmetic commands (e.g.sqr
,sqrt
,cos
, …).
Unfortunately, this indeed optimizes some operations, but it also slows down other a lot (it’s the case for +crop
applied on a large image for instance, which is something done by command pixelsort
.
Hopefully, I’ve been able to write a fix tonight.
I’ll release new version 3.1.4 tomorrow with this fix included.
Thanks @IeBo8Aj for your report, it helps building a better G’MIC !
For more details, see fix here:
I’m wondering if that means you had to undo the optimisation, or if you got the best of both somehow?
Either way, I suppose we’ll find out with the next build…
No, I found a way to keep the optimization for the arithmetic functions, so at the end 3.1.4 will be indeed faster than 3.1.2 & 3.1.3
Nice! Yet another bonus then
Version 3.1.4 of G’MIC has been released, with the fix included.
Thank you @David_Tschumperle for identifying the optimisation problem and releasing a fix so quickly. Thanks too to @afre and @garagecoder for your troubleshooting suggestions.
FWIW, the execution time for my pixelsorting job with GMIC 3.1.3 is 198 seconds, compared with previous times <2 sec. Note that the source image is png
format not jpg
, in case this makes any difference.
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input file 'input.png' at position 0 (1 image 3072x4080x1x3).
[gmic]-1./ Output image [0] as png file 'outfile.png' (1 image 3072x4080x1x3).
[gmic]-1./ End G'MIC interpreter.
gmic input.png fx_pixelsort 1,0,2,1,0,100,0,0,0 output outfile.png 198.80s user 89.66s system 99% cpu 4:49.03 total
I will run this job again after I have installed GMIC 3.1.4 … just waiting for it to hit the arch repositories.
Thanks again, everyone!
Arch packagers are usually very reactive. This should not take too long
The arch repository has now been updated to include GMIC 3.1.4 and GMIC performance is back to normal. I was over-optimistic in my <2 sec execution estimate in the last post … around 5 sec would have been more accurate.
gmic infile.png fx_pixelsort 1,0,2,1,0,100,0,0,0 output outfile.png
6.56s user 0.16s system 105% cpu 6.344 total
This is so much better than yesterday’s 198 seconds!
Thanks again.
I actually think that the command pixelsort
could be optimized a bit.
Not much time to look at this, but I keep this on my TODO.
I reply to myself, but yes, I confirm something faster is possible.
I’ve a prototype of sorting that runs in parallel (e.g. each row sorted in parallel), and I get a x2 factor here. Will try to make this cleaner so I can replace the current pixelsort
function by the parallelized version.
@IeBo8Aj , now that you have G’MIC 3.1.4, could you please try to update your commands ?
$ gmic up
I’ve pushed a modification in the pixelsort
filter that enables parallelization and that should be faster for large images.
Thanks, @David_Tschumperle. I’ve updated G’MIC 3.1.4 as per your instructions and run my script on a test image. It appears to be slightly slower for the same sized image (7.45 sec user versus 6.56 sec user), but I am not using exactly the same file that I used in previous tests, so it is not a properly controlled experiment. (I was using a test file which I have since deleted, thinking I would not be needing it anymore.) Time details below.
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input file 'infile.png' at position 0 (1 image 3072x4080x1x3).
[gmic]-1./ Output image [0] as png file 'outfile.png' (1 image 3072x4080x1x3).
[gmic]-1./ End G'MIC interpreter.
gmic infile.png fx_pixelsort 1,0,2,1,0,100,0,0,0 output outfile.png 7.45s user 0.27s system 141% cpu 5.444 total
It would be helpful if we could test the same image. Please upload and/or send us a link.
With your file, I get this:
$ time gmic infile.png tic fx_pixelsort 1,0,2,1,0,100,0,0,0 toc output outfile.png
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input file 'infile.png' at position 0 (1 image 3072x4080x1x3).
[gmic]-1./ Initialize timer.
[gmic]-1./ Elapsed time: 1.075 s.
[gmic]-1./ Output image [0] as png file 'outfile.png' (1 image 3072x4080x1x3).
[gmic]-1./ End G'MIC interpreter.
real 0m4,494s
user 0m6,398s
sys 0m0,576s
Note that the most time is spent for saving the outfile.png
file. The pixel sorting algorithm takes 1.075s
here. I can’t do anything to speed up the saving, as it is something delegated to the libpng
library.
I hope I didn’t come across as complaining about execution time with the new patch. I’m very happy with the performance, and understand that I/O will be the biggest botteneck.
No worries, but I was trying to figure out why it didn’t seem to be faster for you.
If it’s coming from the I/O, indeed, there is not much we can do.
Thanks again for reporting this regression from 3.1.3!