VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution
This addresses the flexibility and application limitations in video super-resolution for storage-constrained scenarios, though it is incremental as it builds on existing STVSR frameworks.
The paper tackles the problem of fixed up-sampling scales in space-time video super-resolution by proposing VideoINR, which learns an implicit neural representation to decode videos at arbitrary spatial resolutions and frame rates, achieving competitive performance on common scales and significantly outperforming prior works on continuous and out-of-training-distribution scales.
Videos typically record the streaming and continuous visual data as discrete consecutive frames. Since the storage cost is expensive for videos of high fidelity, most of them are stored in a relatively low resolution and frame rate. Recent works of Space-Time Video Super-Resolution (STVSR) are developed to incorporate temporal interpolation and spatial super-resolution in a unified framework. However, most of them only support a fixed up-sampling scale, which limits their flexibility and applications. In this work, instead of following the discrete representations, we propose Video Implicit Neural Representation (VideoINR), and we show its applications for STVSR. The learned implicit neural representation can be decoded to videos of arbitrary spatial resolution and frame rate. We show that VideoINR achieves competitive performances with state-of-the-art STVSR methods on common up-sampling scales and significantly outperforms prior works on continuous and out-of-training-distribution scales. Our project page is at http://zeyuan-chen.com/VideoINR/ .