Video Frame Interpolation by Plug-and-Play Deep Locally Linear Embedding
This addresses video frame interpolation for applications like video editing or compression, offering a plug-and-play, unsupervised method that is incremental in its approach.
The paper tackles video frame interpolation by proposing DeepLLE, a generative framework that uses a deep CNN with a linearity constraint on latent codes to generate new frames, and demonstrates it is highly competitive with state-of-the-art models without requiring large training datasets.
We propose a generative framework which takes on the video frame interpolation problem. Our framework, which we call Deep Locally Linear Embedding (DeepLLE), is powered by a deep convolutional neural network (CNN) while it can be used instantly like conventional models. DeepLLE fits an auto-encoding CNN to a set of several consecutive frames and embeds a linearity constraint on the latent codes so that new frames can be generated by interpolating new latent codes. Different from the current deep learning paradigm which requires training on large datasets, DeepLLE works in a plug-and-play and unsupervised manner, and is able to generate an arbitrary number of frames. Thorough experiments demonstrate that without bells and whistles, our method is highly competitive among current state-of-the-art models.