Humans as Checkerboards: Calibrating Camera Motion Scale for World-Coordinate Human Mesh Recovery
This work addresses a specific bottleneck in 3D human motion estimation from video, offering a significant improvement for applications in robotics and AR/VR.
The paper tackles the problem of unknown scale in monocular SLAM for world-coordinate human mesh recovery by introducing an optimization-free scale calibration framework called Human as Checkerboard (HAC), which reduces motion errors by 50% and uses 100x less inference time than prior methods.
Accurate camera motion estimation is essential for recovering global human motion in world coordinates from RGB video inputs. SLAM is widely used for estimating camera trajectory and point cloud, but monocular SLAM does so only up to an unknown scale factor. Previous works estimate the scale factor through optimization, but this is unreliable and time-consuming. This paper presents an optimization-free scale calibration framework, Human as Checkerboard (HAC). HAC innovatively leverages the human body predicted by human mesh recovery model as a calibration reference. Specifically, it uses the absolute depth of human-scene contact joints as references to calibrate the corresponding relative scene depth from SLAM. HAC benefits from geometric priors encoded in human mesh recovery models to estimate the SLAM scale and achieves precise global human motion estimation. Simple yet powerful, our method sets a new state-of-the-art performance for global human mesh estimation tasks, reducing motion errors by 50% over prior local-to-global methods while using 100$\times$ less inference time than optimization-based methods. Project page: https://martayang.github.io/HAC.