Towards An End-to-End Framework for Flow-Guided Video Inpainting
This work addresses a domain-specific problem for video editing and restoration by making flow-based inpainting more efficient, though it is incremental as it builds on existing flow-guided methods.
The paper tackles the inefficiency and dependency on intermediate stages in flow-guided video inpainting by proposing an end-to-end trainable framework, which outperforms state-of-the-art methods with improved efficiency and effectiveness.
Optical flow, which captures motion information across frames, is exploited in recent video inpainting methods through propagating pixels along its trajectories. However, the hand-crafted flow-based processes in these methods are applied separately to form the whole inpainting pipeline. Thus, these methods are less efficient and rely heavily on the intermediate results from earlier stages. In this paper, we propose an End-to-End framework for Flow-Guided Video Inpainting (E$^2$FGVI) through elaborately designed three trainable modules, namely, flow completion, feature propagation, and content hallucination modules. The three modules correspond with the three stages of previous flow-based methods but can be jointly optimized, leading to a more efficient and effective inpainting process. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods both qualitatively and quantitatively and shows promising efficiency. The code is available at https://github.com/MCG-NKU/E2FGVI.