Deep Frame Interpolation
This work addresses a computer vision problem for applications like cartoon animations, where existing unsupervised methods fail under challenging conditions, but it is incremental as it builds on deep learning and optical flow techniques.
The paper tackles the problem of frame interpolation, especially for low frame rates and large object displacements, by using a deep convolutional neural network and incorporating prior information like optical flow, which significantly improves interpolation quality.
This work presents a supervised learning based approach to the computer vision problem of frame interpolation. The presented technique could also be used in the cartoon animations since drawing each individual frame consumes a noticeable amount of time. The most existing solutions to this problem use unsupervised methods and focus only on real life videos with already high frame rate. However, the experiments show that such methods do not work as well when the frame rate becomes low and object displacements between frames becomes large. This is due to the fact that interpolation of the large displacement motion requires knowledge of the motion structure thus the simple techniques such as frame averaging start to fail. In this work the deep convolutional neural network is used to solve the frame interpolation problem. In addition, it is shown that incorporating the prior information such as optical flow improves the interpolation quality significantly.