LoVoRA: Text-guided and Mask-free Video Object Removal and Addition with Learnable Object-aware Localization
This work addresses the problem of scalable and generalizable video editing for users in multimedia and AI, though it is incremental as it builds on existing text-guided editing methods.
The paper tackles the challenge of text-guided video object removal and addition by proposing LoVoRA, a mask-free framework that achieves temporally consistent edits without auxiliary masks or reference images, demonstrating high-quality performance in experiments and human evaluations.
Text-guided video editing, particularly for object removal and addition, remains a challenging task due to the need for precise spatial and temporal consistency. Existing methods often rely on auxiliary masks or reference images for editing guidance, which limits their scalability and generalization. To address these issues, we propose LoVoRA, a novel framework for mask-free video object removal and addition using object-aware localization mechanism. Our approach utilizes a unique dataset construction pipeline that integrates image-to-video translation, optical flow-based mask propagation, and video inpainting, enabling temporally consistent edits. The core innovation of LoVoRA is its learnable object-aware localization mechanism, which provides dense spatio-temporal supervision for both object insertion and removal tasks. By leveraging a Diffusion Mask Predictor, LoVoRA achieves end-to-end video editing without requiring external control signals during inference. Extensive experiments and human evaluation demonstrate the effectiveness and high-quality performance of LoVoRA. https://cz-5f.github.io/LoVoRA.github.io