DuoMo: Dual Motion Diffusion for World-Space Human Reconstruction
This addresses the challenge of accurate human motion reconstruction for applications like animation or robotics, though it is incremental as it builds on existing diffusion-based methods.
The paper tackles the problem of reconstructing human motion in world-space coordinates from unconstrained videos with noisy or incomplete observations, achieving a 16% reduction in world-space reconstruction error on EMDB and a 30% reduction on RICH while maintaining low foot skating.
We present DuoMo, a generative method that recovers human motion in world-space coordinates from unconstrained videos with noisy or incomplete observations. Reconstructing such motion requires solving a fundamental trade-off: generalizing from diverse and noisy video inputs while maintaining global motion consistency. Our approach addresses this problem by factorizing motion learning into two diffusion models. The camera-space model first estimates motion from videos in camera coordinates. The world-space model then lifts this initial estimate into world coordinates and refines it to be globally consistent. Together, the two models can reconstruct motion across diverse scenes and trajectories, even from highly noisy or incomplete observations. Moreover, our formulation is general, generating the motion of mesh vertices directly and bypassing parametric models. DuoMo achieves state-of-the-art performance. On EMDB, our method obtains a 16% reduction in world-space reconstruction error while maintaining low foot skating. On RICH, it obtains a 30% reduction in world-space error. Project page: https://yufu-wang.github.io/duomo/