Video Restoration with a Deep Plug-and-Play Prior
It addresses video restoration for applications needing high-quality video processing, but is incremental as it adapts existing PnP methods to video.
The paper tackles video restoration by proposing a Deep Plug-and-Play method that uses a video denoising network in an optimization scheme, resulting in better performance and temporal stability compared to image-based or frame-by-frame approaches in tasks like deblurring and super-resolution.
This paper presents a novel method for restoring digital videos via a Deep Plug-and-Play (PnP) approach. Under a Bayesian formalism, the method consists in using a deep convolutional denoising network in place of the proximal operator of the prior in an alternating optimization scheme. We distinguish ourselves from prior PnP work by directly applying that method to restore a digital video from a degraded video observation. This way, a network trained once for denoising can be repurposed for other video restoration tasks. Our experiments in video deblurring, super-resolution, and interpolation of random missing pixels all show a clear benefit to using a network specifically designed for video denoising, as it yields better restoration performance and better temporal stability than a single image network with similar denoising performance using the same PnP formulation. Moreover, our method compares favorably to applying a different state-of-the-art PnP scheme separately on each frame of the sequence. This opens new perspectives in the field of video restoration.