Blind Video Temporal Consistency via Deep Video Prior
This addresses the issue of temporal artifacts in video processing for computer vision applications, but it is incremental as it builds on prior work with a novel training strategy.
The paper tackles the problem of temporal inconsistency in videos when applying image processing algorithms frame-by-frame, and presents a method using Deep Video Prior that achieves superior performance over state-of-the-art methods on 7 computer vision tasks.
Applying image processing algorithms independently to each video frame often leads to temporal inconsistency in the resulting video. To address this issue, we present a novel and general approach for blind video temporal consistency. Our method is only trained on a pair of original and processed videos directly instead of a large dataset. Unlike most previous methods that enforce temporal consistency with optical flow, we show that temporal consistency can be achieved by training a convolutional network on a video with the Deep Video Prior. Moreover, a carefully designed iteratively reweighted training strategy is proposed to address the challenging multimodal inconsistency problem. We demonstrate the effectiveness of our approach on 7 computer vision tasks on videos. Extensive quantitative and perceptual experiments show that our approach obtains superior performance than state-of-the-art methods on blind video temporal consistency. Our source codes are publicly available at github.com/ChenyangLEI/deep-video-prior.