ActionPlan: Future-Aware Streaming Motion Synthesis via Frame-Level Action Planning
This addresses the problem of efficient and high-quality motion synthesis for applications like animation or robotics, though it appears incremental as it builds on existing diffusion methods with a novel workflow.
The paper tackles the challenge of real-time streaming motion synthesis by introducing ActionPlan, a unified motion diffusion framework that uses per-frame action planning to predict text latents as semantic anchors during denoising. As a result, it achieves 5.25x faster real-time streaming and an 18% improvement in motion quality (FID) over previous methods.
We present ActionPlan, a unified motion diffusion framework that bridges real-time streaming with high-quality offline generation within a single model. The core idea is to introduce a per-frame action plan: the model predicts frame-level text latents that act as dense semantic anchors throughout denoising, and uses them to denoise the full motion sequence with combined semantic and motion cues. To support this structured workflow, we design latent-specific diffusion steps, allowing each motion latent to be denoised independently and sampled in flexible orders at inference. As a result, ActionPlan can run in a history-conditioned, future-aware mode for real-time streaming, while also supporting high-quality offline generation. The same mechanism further enables zero-shot motion editing and in-betweening without additional models. Experiments demonstrate that our real-time streaming is 5.25x faster while also achieving 18% motion quality improvement over the best previous method in terms of FID.