Internal Video Inpainting by Implicit Long-range Propagation
This method addresses video inpainting for applications like video editing, offering a novel approach that avoids explicit optical flow, though it appears incremental as it builds on internal learning concepts with added regularizations.
The paper tackles video inpainting by using an internal learning strategy where a convolutional neural network fits known regions to implicitly propagate context across frames, achieving state-of-the-art results on the DAVIS dataset with demonstrated quantitative and qualitative improvements.
We propose a novel framework for video inpainting by adopting an internal learning strategy. Unlike previous methods that use optical flow for cross-frame context propagation to inpaint unknown regions, we show that this can be achieved implicitly by fitting a convolutional neural network to known regions. Moreover, to handle challenging sequences with ambiguous backgrounds or long-term occlusion, we design two regularization terms to preserve high-frequency details and long-term temporal consistency. Extensive experiments on the DAVIS dataset demonstrate that the proposed method achieves state-of-the-art inpainting quality quantitatively and qualitatively. We further extend the proposed method to another challenging task: learning to remove an object from a video giving a single object mask in only one frame in a 4K video.