Flow-Guided Video Inpainting with Scene Templates
This work addresses video inpainting for applications like video editing and restoration, offering a novel method to improve visual quality and reduce distortions, though it is incremental in advancing flow-based techniques.
The paper tackles the problem of filling missing spatio-temporal regions in videos by introducing a flow-based generative model that infers a scene template and mappings to ensure consistency, reducing geometric distortions and artifacts. It demonstrates superior performance over state-of-the-art methods on two benchmark datasets in quantitative metrics and user studies.
We consider the problem of filling in missing spatio-temporal regions of a video. We provide a novel flow-based solution by introducing a generative model of images in relation to the scene (without missing regions) and mappings from the scene to images. We use the model to jointly infer the scene template, a 2D representation of the scene, and the mappings. This ensures consistency of the frame-to-frame flows generated to the underlying scene, reducing geometric distortions in flow based inpainting. The template is mapped to the missing regions in the video by a new L2-L1 interpolation scheme, creating crisp inpaintings and reducing common blur and distortion artifacts. We show on two benchmark datasets that our approach out-performs state-of-the-art quantitatively and in user studies.