Motion-Conditioned Image Animation for Video Editing
This addresses the problem of generating realistic and controllable video edits for users in creative and media fields, representing a strong specific gain rather than a foundational advancement.
The paper tackles video editing by decomposing it into image editing followed by motion-conditioned image animation, introducing MoCA, which achieves a new state-of-the-art with human preference win-rates outperforming methods like Dreamix (63%), MasaCtrl (75%), and Tune-A-Video (72%), especially for motion edits.
We introduce MoCA, a Motion-Conditioned Image Animation approach for video editing. It leverages a simple decomposition of the video editing problem into image editing followed by motion-conditioned image animation. Furthermore, given the lack of robust evaluation datasets for video editing, we introduce a new benchmark that measures edit capability across a wide variety of tasks, such as object replacement, background changes, style changes, and motion edits. We present a comprehensive human evaluation of the latest video editing methods along with MoCA, on our proposed benchmark. MoCA establishes a new state-of-the-art, demonstrating greater human preference win-rate, and outperforming notable recent approaches including Dreamix (63%), MasaCtrl (75%), and Tune-A-Video (72%), with especially significant improvements for motion edits.