TAPE: Temporal Attention-based Probabilistic human pose and shape Estimation
This work addresses challenges in video-based human pose estimation for applications like motion analysis, but it appears incremental as it builds on existing probabilistic and attention-based techniques.
The paper tackles the problem of reconstructing 3D human pose and shape from monocular videos by addressing ambiguities and occlusions, resulting in a method that outperforms state-of-the-art approaches on standard benchmarks.
Reconstructing 3D human pose and shape from monocular videos is a well-studied but challenging problem. Common challenges include occlusions, the inherent ambiguities in the 2D to 3D mapping and the computational complexity of video processing. Existing methods ignore the ambiguities of the reconstruction and provide a single deterministic estimate for the 3D pose. In order to address these issues, we present a Temporal Attention based Probabilistic human pose and shape Estimation method (TAPE) that operates on an RGB video. More specifically, we propose to use a neural network to encode video frames to temporal features using an attention-based neural network. Given these features, we output a per-frame but temporally-informed probability distribution for the human pose using Normalizing Flows. We show that TAPE outperforms state-of-the-art methods in standard benchmarks and serves as an effective video-based prior for optimization-based human pose and shape estimation. Code is available at: https: //github.com/nikosvasilik/TAPE