Motion Manifold Flow Primitives for Task-Conditioned Trajectory Generation under Complex Task-Motion Dependencies
This work addresses the challenge of task-conditioned trajectory generation for robotics and AI systems, particularly in handling many-to-many text-motion correspondences, but it is incremental as it builds on existing motion manifold and flow matching approaches.
The paper tackles the problem of generating trajectories from human demonstrations conditioned on task parameters like language, where existing methods struggle with complex dependencies between tasks and motions. It introduces Motion Manifold Flow Primitives (MMFP), which decouples motion manifold training from task-conditioned distributions using flow matching models, achieving superior performance in language-guided trajectory generation tasks.
Effective movement primitives should be capable of encoding and generating a rich repertoire of trajectories -- typically collected from human demonstrations -- conditioned on task-defining parameters such as vision or language inputs. While recent methods based on the motion manifold hypothesis, which assumes that a set of trajectories lies on a lower-dimensional nonlinear subspace, address challenges such as limited dataset size and the high dimensionality of trajectory data, they often struggle to capture complex task-motion dependencies, i.e., when motion distributions shift drastically with task variations. To address this, we introduce Motion Manifold Flow Primitives (MMFP), a framework that decouples the training of the motion manifold from task-conditioned distributions. Specifically, we employ flow matching models, state-of-the-art conditional deep generative models, to learn task-conditioned distributions in the latent coordinate space of the learned motion manifold. Experiments are conducted on language-guided trajectory generation tasks, where many-to-many text-motion correspondences introduce complex task-motion dependencies, highlighting MMFP's superiority over existing methods.