Cross View Fusion for 3D Human Pose Estimation
This work addresses the problem of accurate 3D human pose estimation for computer vision applications, showing incremental improvements over existing methods.
The paper tackles 3D human pose estimation from multi-view images by introducing a cross-view fusion scheme for 2D pose estimation and a recursive Pictorial Structure Model for 3D recovery, achieving Mean Per Joint Position Errors of 26mm and 29mm on H36M and Total Capture datasets, outperforming state-of-the-art methods.
We present an approach to recover absolute 3D human poses from multi-view images by incorporating multi-view geometric priors in our model. It consists of two separate steps: (1) estimating the 2D poses in multi-view images and (2) recovering the 3D poses from the multi-view 2D poses. First, we introduce a cross-view fusion scheme into CNN to jointly estimate 2D poses for multiple views. Consequently, the 2D pose estimation for each view already benefits from other views. Second, we present a recursive Pictorial Structure Model to recover the 3D pose from the multi-view 2D poses. It gradually improves the accuracy of 3D pose with affordable computational cost. We test our method on two public datasets H36M and Total Capture. The Mean Per Joint Position Errors on the two datasets are 26mm and 29mm, which outperforms the state-of-the-arts remarkably (26mm vs 52mm, 29mm vs 35mm). Our code is released at \url{https://github.com/microsoft/multiview-human-pose-estimation-pytorch}.