MotionGrounder: Grounded Multi-Object Motion Transfer via Diffusion Transformer
This work addresses the limitation of single-object control in motion transfer for real-world scenes with multiple objects, offering a solution for fine-grained video generation.
The paper tackles the problem of multi-object motion transfer in video generation, introducing MotionGrounder, a Diffusion Transformer-based framework that achieves superior performance in quantitative, qualitative, and human evaluations compared to existing baselines.
Motion transfer enables controllable video generation by transferring temporal dynamics from a reference video to synthesize a new video conditioned on a target caption. However, existing Diffusion Transformer (DiT)-based methods are limited to single-object videos, restricting fine-grained control in real-world scenes with multiple objects. In this work, we introduce MotionGrounder, a DiT-based framework that firstly handles motion transfer with multi-object controllability. Our Flow-based Motion Signal (FMS) in MotionGrounder provides a stable motion prior for target video generation, while our Object-Caption Alignment Loss (OCAL) grounds object captions to their corresponding spatial regions. We further propose a new Object Grounding Score (OGS), which jointly evaluates (i) spatial alignment between source video objects and their generated counterparts and (ii) semantic consistency between each generated object and its target caption. Our experiments show that MotionGrounder consistently outperforms recent baselines across quantitative, qualitative, and human evaluations.