That is something GāMIC should be able to do also. Give me 5 minutesā¦
I do remember seeing the optical flow estimation at some point. It would be really cool if we could get GāMIC to do that!
Actually, command -morph
already does that, but itās a bit memory consuming right now (because it keeps all interpolated frames in memory). So, Iām working on a āstreamedā version of -morph
.
Thanks for the feedback. Iāll let you know when Iāve got a proper edit done which I think will contain some other sequences, too.
I see - thanks.
Thatās the one I initially tried, but I found that sometimes some stars would be moved along with the lights and then snap back into place on the next ākeyframeā. Iād definitely be up for testing a streamed version of GāMICās morph!
OK, so Iāve added another set of commands -morph_files
and -morph_video
which basically works like fade_files
and fade_videos
, with some extra parameters for the morphing algorithm.
The morphing is done between two consecutive frames by estimating both the forward and backward motion vectors (using an optical-flow like method), then interpolate the frames temporally using these two motion vector fields.
No need to say this can be very time consuming ! (even if like me you have 24 cores!)
How to use it?
-
First, update your filters, and check the commands are recognized:
gmic -update gmic -h morph_files
...(help should display here)...
An example of use, using image files as the input (here again, using bash on Linux, may be slightly different on Windows):
$ gmic -w -morph_files input_images\*.jpg,20,0.1,4,0,-1,1,output.png
(you may want to replace output.png
by output.avi
to gets an .avi
video file as the output).
And thatās almost the same if you have an input video file, instead of a sequence of images:
$ gmic -w --morph_video input_video.avi,20,0.1,4,0,-1,1,output.png
The smoothness
parameter is important to set correctly for the morphing algorithm. Basically if the frames you want to interpolate can be well registered by a rigid motion (translation,shift,ā¦) then the smoothness
can be high (like 1 or 1.5). For non-rigid motions, try a lower value (0.1 is medium, 0.01 is low).
After using the update command, the morph_files
command doesnāt seem to be recognised.
$ gmic -update
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Update commands from the latest definition file on the G'MIC server.
[gmic]-0./ End G'MIC interpreter.
$ gmic -h morph_files
gmic: GREYC's Magic for Image Computing.
Version 1.7.3, Copyright (c) 2008-2016, David Tschumperle.
(http://gmic.eu)
[gmic] Command 'morph_files' has no description. Try 'gmic -h' for global help.
hum. Isnāt it because of some caching problems ?
Try to get the update file directly from http://gmic.eu/update174.gmic, and put it in your $HOME/.config/gmic/
folder (replacing the older one).
Here is an example of the (subtle) differences you get with commands -fade_video
and -morph_video
.
Not sure the morphing here adds a lot (except for the computation time of course ).
Adding that file to GāMICās config directory has unfortunately not solved the issue. I also tried adding cli_update174.gmic
.
Thatās a nice comparison video. The morphing really does help the smoothness in many places.
Ah yes, sorry but if you have version 1.7.3 of GāMIC, you need to rename this file $HOME/.config/gmic/cli_update173.gmic
. It should work then (but in this case, donāt invoke $ gmic -update
, otherwise the cli_update file173.gmic
will be overwritten.
I see - thanks. Iāve now downloaded the following files and replaced ā174ā with ā173ā.
gmic -h morph_files
is still not showing help for the command, and gmic --morph_files
gives:
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ *** Warning *** File '/home/davido/.config/gmic/cli_update173.gmic' is not a valid G'MIC command file.
[gmic]-0./ Input file '--morph_files' at position 0
[gmic]-0./ *** Error *** Unknown command or filename '--morph_files'.
It seems like Iāll need to update GāMIC in order to test?
Edit: should the morph_files
command be preprended by one or two dashes?
Well, no, the warning you get is not good :
[gmic]-0./ *** Warning *** File ā/home/davido/.config/gmic/cli_update173.gmicā is not a valid GāMIC command file.
I think you should try $ gmic -update
again, this is the āgoodā way to update. Then, check if the file $HOME/.config/gmic/cli_update173.gmic
is a plain text file, and contains the string morph_stream
.
If it doesnāt then it means you didnāt get the latest updates, for some strange reasons (maybe web cachingā¦).
After deleting the update files and trying gmic -update
again, all seems well. Thanks!
Iāll leave it processing overnight and compare in the morning.
@David_Tschumperle found a pretty cool use for this. It works quite well for interpolating in between the frames of precipitation radar sequences. Not yet sure if I can move this into production but Iāll let you know if I can.
Morph, 7 intra-frames, 60fps, 1.25 smoothing
(I found low smoothing values resulted in stars being shifted.)
Crossfade, 7 intra-frames, 60fps
Thoughts
- I donāt notice much difference in smoothness between the two in this case; I suspect greater movement between each exposure and/or a slower playback speed would show more of a difference. Iāll experiment with this further.
- By adjusting the smoothing value I got much, much less unwanted star movement than when using slowmoVideo. (I was using the CPU option with slowmoVideo, not the GPU option which doesnāt seem to be available in my current system setup.)
- There is one point in the morph version, shortly after 1:10, where the middle of the horizon twitches; I think this was triggered by inconsistent colours of three dodgy exposures where someone (possibly me) was shining a torch where they shouldnāt have been. For series such as this where crossfading is sufficient Iāll probably stick with the crossfading to avoid unwanted movement, but will try morphing when the combination of exposure time and playback speed result in substantial movement between ākeyframesā.
- With my 796 source images, using the morph option with 7 intra-frames resulted in more images (6,361) than using the crossfade option with 7 intra-frames (5,565). Iāll do some experimenting with this on a smaller number of images to see if this is consistently the case.
- @David_Tschumperle is magnifique.
I played with this applied to some mobile phone slow motion footage. Here is my result:
Thank you @David_Tschumperle
That came out great. How many frames did you end up with for each lightning bolt? Thanks
I donāt remember but probably something like 5.
I know itās been a while on this topic, but I just came across this work.
The video samples look incredible.
I canāt get it running on my windows machine, but maybe you will have better luck. Maybe itāll make it into GāMIC!
LOL All want to gmic + neural network