A Simple Baseline for Video Restoration with Grouped Spatial-temporal Shift
This work addresses the high computational cost in video restoration for applications requiring efficient processing, though it is incremental as it builds on existing shift-based techniques.
The authors tackled video restoration by proposing a simple framework using grouped spatial-temporal shift to capture inter-frame information, which outperformed the previous state-of-the-art method with less than a quarter of the computational cost in deblurring and denoising tasks.
Video restoration, which aims to restore clear frames from degraded videos, has numerous important applications. The key to video restoration depends on utilizing inter-frame information. However, existing deep learning methods often rely on complicated network architectures, such as optical flow estimation, deformable convolution, and cross-frame self-attention layers, resulting in high computational costs. In this study, we propose a simple yet effective framework for video restoration. Our approach is based on grouped spatial-temporal shift, which is a lightweight and straightforward technique that can implicitly capture inter-frame correspondences for multi-frame aggregation. By introducing grouped spatial shift, we attain expansive effective receptive fields. Combined with basic 2D convolution, this simple framework can effectively aggregate inter-frame information. Extensive experiments demonstrate that our framework outperforms the previous state-of-the-art method, while using less than a quarter of its computational cost, on both video deblurring and video denoising tasks. These results indicate the potential for our approach to significantly reduce computational overhead while maintaining high-quality results. Code is avaliable at https://github.com/dasongli1/Shift-Net.