Efficient Feature Extraction for High-resolution Video Frame Interpolation
This addresses the challenge of efficient feature extraction for high-resolution video interpolation, offering a solution that reduces computational and memory costs for applications in video processing.
The paper tackles the problem of high-resolution video frame interpolation by proposing a method that compresses input representations using dimensionality reduction and lightweight optimization, achieving state-of-the-art image quality on 4K benchmarks with the lowest network complexity and memory requirements.
Most deep learning methods for video frame interpolation consist of three main components: feature extraction, motion estimation, and image synthesis. Existing approaches are mainly distinguishable in terms of how these modules are designed. However, when interpolating high-resolution images, e.g. at 4K, the design choices for achieving high accuracy within reasonable memory requirements are limited. The feature extraction layers help to compress the input and extract relevant information for the latter stages, such as motion estimation. However, these layers are often costly in parameters, computation time, and memory. We show how ideas from dimensionality reduction combined with a lightweight optimization can be used to compress the input representation while keeping the extracted information suitable for frame interpolation. Further, we require neither a pretrained flow network nor a synthesis network, additionally reducing the number of trainable parameters and required memory. When evaluating on three 4K benchmarks, we achieve state-of-the-art image quality among the methods without pretrained flow while having the lowest network complexity and memory requirements overall.