HULC: 3D Human Motion Capture with Pose Manifold Sampling and Dense Contact Guidance
This addresses the challenge of accurate 3D human motion capture for applications in extended reality, robotics, and virtual avatar generation, representing a strong specific gain in the field.
The paper tackled the problem of marker-less monocular 3D human motion capture with scene interactions, which often suffers from artefacts like inter-penetrations and jitter, by proposing HULC, a method that estimates 3D poses, dense contacts, and absolute scale, resulting in significantly more physically-plausible poses that outperform existing approaches in various experiments.
Marker-less monocular 3D human motion capture (MoCap) with scene interactions is a challenging research topic relevant for extended reality, robotics and virtual avatar generation. Due to the inherent depth ambiguity of monocular settings, 3D motions captured with existing methods often contain severe artefacts such as incorrect body-scene inter-penetrations, jitter and body floating. To tackle these issues, we propose HULC, a new approach for 3D human MoCap which is aware of the scene geometry. HULC estimates 3D poses and dense body-environment surface contacts for improved 3D localisations, as well as the absolute scale of the subject. Furthermore, we introduce a 3D pose trajectory optimisation based on a novel pose manifold sampling that resolves erroneous body-environment inter-penetrations. Although the proposed method requires less structured inputs compared to existing scene-aware monocular MoCap algorithms, it produces more physically-plausible poses: HULC significantly and consistently outperforms the existing approaches in various experiments and on different metrics. Project page: https://vcai.mpi-inf.mpg.de/projects/HULC/.