A generic diffusion-based approach for 3D human pose prediction in the wild
This addresses the challenge of noisy pose estimation for applications like robotics and AR/VR, though it is incremental as it adapts diffusion models to an existing task.
The paper tackles the problem of predicting 3D human poses from noisy inputs in real-world scenarios by proposing a diffusion-based approach that frames prediction as a denoising task, outperforming state-of-the-art methods on four public datasets.
Predicting 3D human poses in real-world scenarios, also known as human pose forecasting, is inevitably subject to noisy inputs arising from inaccurate 3D pose estimations and occlusions. To address these challenges, we propose a diffusion-based approach that can predict given noisy observations. We frame the prediction task as a denoising problem, where both observation and prediction are considered as a single sequence containing missing elements (whether in the observation or prediction horizon). All missing elements are treated as noise and denoised with our conditional diffusion model. To better handle long-term forecasting horizon, we present a temporal cascaded diffusion model. We demonstrate the benefits of our approach on four publicly available datasets (Human3.6M, HumanEva-I, AMASS, and 3DPW), outperforming the state-of-the-art. Additionally, we show that our framework is generic enough to improve any 3D pose prediction model as a pre-processing step to repair their inputs and a post-processing step to refine their outputs. The code is available online: \url{https://github.com/vita-epfl/DePOSit}.