Towards Geometry-Aware and Motion-Guided Video Human Mesh Recovery
This work addresses physically implausible 3D human mesh recovery in videos, which is crucial for applications in animation, virtual reality, and robotics, representing a significant advancement rather than an incremental improvement.
The paper tackled the problem of video-based 3D Human Mesh Recovery (HMR) producing physically implausible results by introducing HMRMamba, a new paradigm using Structured State Space Models (SSMs), which achieved state-of-the-art performance on benchmarks like 3DPW, MPI-INF-3DHP, and Human3.6M, improving reconstruction accuracy and temporal consistency with superior computational efficiency.
Existing video-based 3D Human Mesh Recovery (HMR) methods often produce physically implausible results, stemming from their reliance on flawed intermediate 3D pose anchors and their inability to effectively model complex spatiotemporal dynamics. To overcome these deep-rooted architectural problems, we introduce HMRMamba, a new paradigm for HMR that pioneers the use of Structured State Space Models (SSMs) for their efficiency and long-range modeling prowess. Our framework is distinguished by two core contributions. First, the Geometry-Aware Lifting Module, featuring a novel dual-scan Mamba architecture, creates a robust foundation for reconstruction. It directly grounds the 2D-to-3D pose lifting process with geometric cues from image features, producing a highly reliable 3D pose sequence that serves as a stable anchor. Second, the Motion-guided Reconstruction Network leverages this anchor to explicitly process kinematic patterns over time. By injecting this crucial temporal awareness, it significantly enhances the final mesh's coherence and robustness, particularly under occlusion and motion blur. Comprehensive evaluations on 3DPW, MPI-INF-3DHP, and Human3.6M benchmarks confirm that HMRMamba sets a new state-of-the-art, outperforming existing methods in both reconstruction accuracy and temporal consistency while offering superior computational efficiency.