How can I interpolate or crossfade a series of images?

[David_Tschumperle (G'MIC staff)]

That is something G’MIC should be able to do also. Give me 5 minutes…

[Jonas_Wagner]

I do remember seeing the optical flow estimation at some point. It would be really cool if we could get G’MIC to do that!

[David_Tschumperle (G'MIC staff)]

Actually, command -morph already does that, but it’s a bit memory consuming right now (because it keeps all interpolated frames in memory). So, I’m working on a ‘streamed’ version of -morph.

[DavidOliver]

Thanks for the feedback. I’ll let you know when I’ve got a proper edit done which I think will contain some other sequences, too.

@David_Tschumperle

The frame_step parameter can be used to ‘skip’ some frames for the computation

I see - thanks.

@Jonas_Wagner

http://slowmovideo.granjow.net/ might be worth trying too.

That’s the one I initially tried, but I found that sometimes some stars would be moved along with the lights and then snap back into place on the next “keyframe”. I’d definitely be up for testing a streamed version of G’MIC’s morph!

[David_Tschumperle (G'MIC staff)]

OK, so I’ve added another set of commands -morph_files and -morph_video which basically works like fade_files and fade_videos, with some extra parameters for the morphing algorithm.
The morphing is done between two consecutive frames by estimating both the forward and backward motion vectors (using an optical-flow like method), then interpolate the frames temporally using these two motion vector fields.
No need to say this can be very time consuming ! (even if like me you have 24 cores!)

How to use it?

• First, update your filters, and check the commands are recognized:

  gmic -update gmic -h morph_files

   ...(help should display here)...
  

An example of use, using image files as the input (here again, using bash on Linux, may be slightly different on Windows):

$ gmic -w -morph_files input_images\*.jpg,20,0.1,4,0,-1,1,output.png

(you may want to replace output.png by output.avi to gets an .avi video file as the output).
And that’s almost the same if you have an input video file, instead of a sequence of images:

$ gmic -w --morph_video input_video.avi,20,0.1,4,0,-1,1,output.png

The smoothness parameter is important to set correctly for the morphing algorithm. Basically if the frames you want to interpolate can be well registered by a rigid motion (translation,shift,…) then the smoothness can be high (like 1 or 1.5). For non-rigid motions, try a lower value (0.1 is medium, 0.01 is low).

[DavidOliver]

After using the update command, the morph_files command doesn’t seem to be recognised.

$ gmic -update
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Update commands from the latest definition file on the G'MIC server.
[gmic]-0./ End G'MIC interpreter.
$ gmic -h morph_files
  gmic: GREYC's Magic for Image Computing. 
 
        Version 1.7.3, Copyright (c) 2008-2016, David Tschumperle. 
        (http://gmic.eu)

[gmic] Command 'morph_files' has no description. Try 'gmic -h' for global help.

[David_Tschumperle (G'MIC staff)]

hum. Isn’t it because of some caching problems ?
Try to get the update file directly from http://gmic.eu/update174.gmic, and put it in your $HOME/.config/gmic/ folder (replacing the older one).

[David_Tschumperle (G'MIC staff)]

Here is an example of the (subtle) differences you get with commands -fade_video and -morph_video.

Difference between temporal fading and morphing

Not sure the morphing here adds a lot (except for the computation time of course ).

[DavidOliver]

Adding that file to G’MIC’s config directory has unfortunately not solved the issue. I also tried adding cli_update174.gmic.

That’s a nice comparison video. The morphing really does help the smoothness in many places.

[David_Tschumperle (G'MIC staff)]

Ah yes, sorry but if you have version 1.7.3 of G’MIC, you need to rename this file $HOME/.config/gmic/cli_update173.gmic. It should work then (but in this case, don’t invoke $ gmic -update, otherwise the cli_update file173.gmic will be overwritten.

[DavidOliver]

I see - thanks. I’ve now downloaded the following files and replaced ‘174’ with ‘173’.

• http://gmic.eu/update174.gmic
• http://gmic.eu/cli_update174.gmic

gmic -h morph_files is still not showing help for the command, and gmic --morph_files gives:

[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ *** Warning *** File '/home/davido/.config/gmic/cli_update173.gmic' is not a valid G'MIC command file.
[gmic]-0./ Input file '--morph_files' at position 0
[gmic]-0./ *** Error *** Unknown command or filename '--morph_files'.

It seems like I’ll need to update G’MIC in order to test?

Edit: should the morph_files command be preprended by one or two dashes?

[David_Tschumperle (G'MIC staff)]

Well, no, the warning you get is not good :

[gmic]-0./ *** Warning *** File ‘/home/davido/.config/gmic/cli_update173.gmic’ is not a valid G’MIC command file.

I think you should try $ gmic -update again, this is the ‘good’ way to update. Then, check if the file $HOME/.config/gmic/cli_update173.gmic is a plain text file, and contains the string morph_stream.
If it doesn’t then it means you didn’t get the latest updates, for some strange reasons (maybe web caching…).

[DavidOliver]

After deleting the update files and trying gmic -update again, all seems well. Thanks!

I’ll leave it processing overnight and compare in the morning.

[Jonas_Wagner]

@David_Tschumperle found a pretty cool use for this. It works quite well for interpolating in between the frames of precipitation radar sequences. Not yet sure if I can move this into production but I’ll let you know if I can.

[DavidOliver]

Morph, 7 intra-frames, 60fps, 1.25 smoothing

(I found low smoothing values resulted in stars being shifted.)

Test: Morph: Northern Lights 1, Iceland, 2016

Crossfade, 7 intra-frames, 60fps

https://youtu.be/sYixTgSZw28

Thoughts

• I don’t notice much difference in smoothness between the two in this case; I suspect greater movement between each exposure and/or a slower playback speed would show more of a difference. I’ll experiment with this further.
• By adjusting the smoothing value I got much, much less unwanted star movement than when using slowmoVideo. (I was using the CPU option with slowmoVideo, not the GPU option which doesn’t seem to be available in my current system setup.)
• There is one point in the morph version, shortly after 1:10, where the middle of the horizon twitches; I think this was triggered by inconsistent colours of three dodgy exposures where someone (possibly me) was shining a torch where they shouldn’t have been. For series such as this where crossfading is sufficient I’ll probably stick with the crossfading to avoid unwanted movement, but will try morphing when the combination of exposure time and playback speed result in substantial movement between “keyframes”.
• With my 796 source images, using the morph option with 7 intra-frames resulted in more images (6,361) than using the crossfade option with 7 intra-frames (5,565). I’ll do some experimenting with this on a smaller number of images to see if this is consistently the case.
• @David_Tschumperle is magnifique.

[Jonas_Wagner]

I played with this applied to some mobile phone slow motion footage. Here is my result:

Thank you @David_Tschumperle

[harry_durgin]

That came out great. How many frames did you end up with for each lightning bolt? Thanks

[Jonas_Wagner]

I don’t remember but probably something like 5.

[n0tn0 (Nathan Rosquist)]

I know it’s been a while on this topic, but I just came across this work.

The video samples look incredible.

I can’t get it running on my windows machine, but maybe you will have better luck. Maybe it’ll make it into G’MIC!

@DavidOliver @David_Tschumperle

[bazza (http://4232.cf)]

LOL All want to gmic + neural network