Neural Compression-Based Feature Learning for Video Restoration
This work addresses video restoration for applications like denoising, deraining, and dehazing, offering an incremental improvement by integrating neural compression into feature learning.
The paper tackles the challenge of noisy and uncorrelated temporal features in video restoration by proposing a neural compression module that filters noise to improve performance. It achieves a 0.13 dB improvement in video denoising over BasicVSR++ with only 0.23x FLOPs and state-of-the-art results in deraining and dehazing.
How to efficiently utilize the temporal features is crucial, yet challenging, for video restoration. The temporal features usually contain various noisy and uncorrelated information, and they may interfere with the restoration of the current frame. This paper proposes learning noise-robust feature representations to help video restoration. We are inspired by that the neural codec is a natural denoiser. In neural codec, the noisy and uncorrelated contents which are hard to predict but cost lots of bits are more inclined to be discarded for bitrate saving. Therefore, we design a neural compression module to filter the noise and keep the most useful information in features for video restoration. To achieve robustness to noise, our compression module adopts a spatial channel-wise quantization mechanism to adaptively determine the quantization step size for each position in the latent. Experiments show that our method can significantly boost the performance on video denoising, where we obtain 0.13 dB improvement over BasicVSR++ with only 0.23x FLOPs. Meanwhile, our method also obtains SOTA results on video deraining and dehazing.