Characterizing Motion Encoding in Video Diffusion Timesteps
This work provides a systematic characterization of motion encoding for video diffusion practitioners, turning a heuristic into a principle for spatiotemporal disentanglement, though it is incremental as it builds on existing models.
The paper tackled the problem of understanding how motion is encoded across timesteps in text-to-video diffusion models, finding consistent early motion-dominant and later appearance-dominant regimes through a large-scale quantitative study. It simplified motion customization by restricting training and inference to the motion-dominant regime, achieving strong motion transfer without extra modules.
Text-to-video diffusion models synthesize temporal motion and spatial appearance through iterative denoising, yet how motion is encoded across timesteps remains poorly understood. Practitioners often exploit the empirical heuristic that early timesteps mainly shape motion and layout while later ones refine appearance, but this behavior has not been systematically characterized. In this work, we proxy motion encoding in video diffusion timesteps by the trade-off between appearance editing and motion preservation induced when injecting new conditions over specified timestep ranges, and characterize this proxy through a large-scale quantitative study. This protocol allows us to factor motion from appearance by quantitatively mapping how they compete along the denoising trajectory. Across diverse architectures, we consistently identify an early, motion-dominant regime and a later, appearance-dominant regime, yielding an operational motion-appearance boundary in timestep space. Building on this characterization, we simplify current one-shot motion customization paradigm by restricting training and inference to the motion-dominant regime, achieving strong motion transfer without auxiliary debiasing modules or specialized objectives. Our analysis turns a widely used heuristic into a spatiotemporal disentanglement principle, and our timestep-constrained recipe can serve as ready integration into existing motion transfer and editing methods.