Learning Long-Term Style-Preserving Blind Video Temporal Consistency
This addresses the issue of temporal artifacts in video processing for applications like style transfer, though it is incremental as it builds on existing post-processing techniques.
The paper tackles the problem of flickering artifacts when applying image-trained algorithms to video frames, proposing a post-processing model that improves long-term temporal consistency and better preserves video style compared to previous methods, achieving state-of-the-art results on datasets like DAVIS and videvo.net.
When trying to independently apply image-trained algorithms to successive frames in videos, noxious flickering tends to appear. State-of-the-art post-processing techniques that aim at fostering temporal consistency, generate other temporal artifacts and visually alter the style of videos. We propose a postprocessing model, agnostic to the transformation applied to videos (e.g. style transfer, image manipulation using GANs, etc.), in the form of a recurrent neural network. Our model is trained using a Ping Pong procedure and its corresponding loss, recently introduced for GAN video generation, as well as a novel style preserving perceptual loss. The former improves long-term temporal consistency learning, while the latter fosters style preservation. We evaluate our model on the DAVIS and videvo.net datasets and show that our approach offers state-of-the-art results concerning flicker removal, and better keeps the overall style of the videos than previous approaches.