MotionShot: Adaptive Motion Transfer across Arbitrary Objects for Text-to-Video Generation
This addresses a challenge in text-to-video generation for applications requiring smooth motion transfer across arbitrary objects, though it appears incremental as it builds on existing methods.
The paper tackles the problem of transferring motion from a reference object to a target object with different appearances or structures in text-to-video generation, achieving high-fidelity motion transfer while preserving coherence, as demonstrated by extensive experiments.
Existing text-to-video methods struggle to transfer motion smoothly from a reference object to a target object with significant differences in appearance or structure between them. To address this challenge, we introduce MotionShot, a training-free framework capable of parsing reference-target correspondences in a fine-grained manner, thereby achieving high-fidelity motion transfer while preserving coherence in appearance. To be specific, MotionShot first performs semantic feature matching to ensure high-level alignments between the reference and target objects. It then further establishes low-level morphological alignments through reference-to-target shape retargeting. By encoding motion with temporal attention, our MotionShot can coherently transfer motion across objects, even in the presence of significant appearance and structure disparities, demonstrated by extensive experiments. The project page is available at: https://motionshot.github.io/.