Revisiting Temporal Modeling for Video Super-resolution
This work addresses the need for reliable evaluation in video super-resolution for applications like surveillance and UHD display, though it is incremental as it builds on existing temporal modeling approaches.
The authors tackled the problem of inconsistent comparisons in video super-resolution by systematically evaluating three temporal modeling methods and proposing a Recurrent Residual Network (RRN), which achieved state-of-the-art results on benchmarks with high computational efficiency and finer details.
Video super-resolution plays an important role in surveillance video analysis and ultra-high-definition video display, which has drawn much attention in both the research and industrial communities. Although many deep learning-based VSR methods have been proposed, it is hard to directly compare these methods since the different loss functions and training datasets have a significant impact on the super-resolution results. In this work, we carefully study and compare three temporal modeling methods (2D CNN with early fusion, 3D CNN with slow fusion and Recurrent Neural Network) for video super-resolution. We also propose a novel Recurrent Residual Network (RRN) for efficient video super-resolution, where residual learning is utilized to stabilize the training of RNN and meanwhile to boost the super-resolution performance. Extensive experiments show that the proposed RRN is highly computational efficiency and produces temporal consistent VSR results with finer details than other temporal modeling methods. Besides, the proposed method achieves state-of-the-art results on several widely used benchmarks.