Deep Motion Blind Video Stabilization
This work provides a new dataset and a faster, purely regressive deep-learning approach to video stabilization, benefiting users and applications requiring efficient and high-quality video stabilization.
This paper addresses the lack of a purely regressive deep-learning formulation for video stabilization by introducing a novel dataset of video pairs with similar perspectives but different motions. Using this dataset, they successfully train a motion-blind, full-frame video stabilization model with conventional generative techniques, achieving a ~3x speed-up over existing state-of-the-art methods.
Despite the advances in the field of generative models in computer vision, video stabilization still lacks a pure regressive deep-learning-based formulation. Deep video stabilization is generally formulated with the help of explicit motion estimation modules due to the lack of a dataset containing pairs of videos with similar perspective but different motion. Therefore, the deep learning approaches for this task have difficulties in the pixel-level synthesis of latent stabilized frames, and resort to motion estimation modules for indirect transformations of the unstable frames to stabilized frames, leading to the loss of visual content near the frame boundaries. In this work, we aim to declutter this over-complicated formulation of video stabilization with the help of a novel dataset that contains pairs of training videos with similar perspective but different motion, and verify its effectiveness by successfully learning motion blind full-frame video stabilization through employing strictly conventional generative techniques and further improve the stability through a curriculum-learning inspired adversarial training strategy. Through extensive experimentation, we show the quantitative and qualitative advantages of the proposed approach to the state-of-the-art video stabilization approaches. Moreover, our method achieves $\sim3\times$ speed-up over the currently available fastest video stabilization methods.