SnapPose3D: Diffusion-Based Single-Frame 2D-to-3D Lifting of Human Poses
For researchers in 3D human pose estimation, SnapPose3D provides a novel single-frame method that outperforms previous temporal-sequence approaches, enabling real-time applications without tracking.
SnapPose3D addresses depth ambiguity and joint uncertainty in 2D-to-3D human pose lifting by using a diffusion-based model to generate multiple hypotheses and aggregate them into a final accurate pose, achieving state-of-the-art results on standard benchmarks with single-frame input.
Depth ambiguity and joint uncertainty are the two main obstacles in obtaining accurate human pose predictions by 2D-to-3D lifting methods proposed in the literature. In particular, these issues are caused by 2D joint locations that can be mapped to multiple 3D positions, inducing multiple possible final poses. Following these considerations, we propose leveraging diffusion-based models generation capability to predict multiple hypotheses and aggregate them in a final accurate pose. Therefore, we introduce SnapPose3D, a pose-lifting framework trained deterministically to denoise 3D poses conditioned on both visual context and 2D pose features. SnapPose3D adopts a probabilistic approach during inference, generating multiple hypotheses through random sampling from a unit Gaussian distribution. Unlike most previous methods that address pose ambiguity by processing temporal sequences, SnapPose3D uses single frames as input, avoiding tracking and limiting computational cost, data acquisition complexity, and the need for online, real-time applications. We extensively evaluate SnapPose3D on well-known benchmarks for the 3D human pose estimation task showing its ability to generate and aggregate accurate hypotheses that lead to state-of-the-art results.