SMPLy Benchmarking 3D Human Pose Estimation in the Wild
This work provides a much-needed benchmark for 3D human pose estimation in unconstrained, in-the-wild conditions, which is crucial for researchers developing methods for real-world applications.
This paper introduces a new pipeline to create a dataset of 24,428 frames with registered SMPL body models from 567 in-the-wild video scenes. It then benchmarks state-of-the-art SMPL-based 3D human pose estimation methods on this dataset, revealing remaining challenges in difficult poses and occluded scenes.
Predicting 3D human pose from images has seen great recent improvements. Novel approaches that can even predict both pose and shape from a single input image have been introduced, often relying on a parametric model of the human body such as SMPL. While qualitative results for such methods are often shown for images captured in-the-wild, a proper benchmark in such conditions is still missing, as it is cumbersome to obtain ground-truth 3D poses elsewhere than in a motion capture room. This paper presents a pipeline to easily produce and validate such a dataset with accurate ground-truth, with which we benchmark recent 3D human pose estimation methods in-the-wild. We make use of the recently introduced Mannequin Challenge dataset which contains in-the-wild videos of people frozen in action like statues and leverage the fact that people are static and the camera moving to accurately fit the SMPL model on the sequences. A total of 24,428 frames with registered body models are then selected from 567 scenes at almost no cost, using only online RGB videos. We benchmark state-of-the-art SMPL-based human pose estimation methods on this dataset. Our results highlight that challenges remain, in particular for difficult poses or for scenes where the persons are partially truncated or occluded.