Free3D: 3D Human Motion Emerges from Single-View 2D Supervision
This addresses the need for scalable and data-efficient 3D motion generation in computer vision and robotics, offering a novel approach that reduces reliance on costly 3D data.
The paper tackles the problem of 3D human motion generation models struggling to generalize beyond training distributions by proposing Free3D, a framework that synthesizes realistic 3D motions without 3D annotations, achieving performance comparable to or surpassing fully 3D-supervised models.
Recent 3D human motion generation models demonstrate remarkable reconstruction accuracy yet struggle to generalize beyond training distributions. This limitation arises partly from the use of precise 3D supervision, which encourages models to fit fixed coordinate patterns instead of learning the essential 3D structure and motion semantic cues required for robust generalization.To overcome this limitation, we propose Free3D, a framework that synthesizes realistic 3D motions without any 3D motion annotations. Free3D introduces a Motion-Lifting Residual Quantized VAE (ML-RQ) that maps 2D motion sequences into 3D-consistent latent spaces, and a suite of 3D-free regularization objectives enforcing view consistency, orientation coherence, and physical plausibility. Trained entirely on 2D motion data, Free3D generates diverse, temporally coherent, and semantically aligned 3D motions, achieving performance comparable to or even surpassing fully 3D-supervised counterparts. These results suggest that relaxing explicit 3D supervision encourages stronger structural reasoning and generalization, offering a scalable and data-efficient paradigm for 3D motion generation.