A Single 2D Pose with Context is Worth Hundreds for 3D Human Pose Estimation
This work improves 3D human pose estimation for computer vision applications by reducing computational demands and enhancing accuracy, representing a novel approach rather than an incremental improvement.
The paper tackles the problem of 3D human pose estimation by addressing the reliance on long-term temporal clues in existing methods, which causes performance saturation and high computation. The proposed solution uses intermediate visual representations from pre-trained 2D pose detectors to encode spatial context, resulting in a method that significantly outperforms state-of-the-art approaches in both speed and precision without using temporal information.
The dominant paradigm in 3D human pose estimation that lifts a 2D pose sequence to 3D heavily relies on long-term temporal clues (i.e., using a daunting number of video frames) for improved accuracy, which incurs performance saturation, intractable computation and the non-causal problem. This can be attributed to their inherent inability to perceive spatial context as plain 2D joint coordinates carry no visual cues. To address this issue, we propose a straightforward yet powerful solution: leveraging the readily available intermediate visual representations produced by off-the-shelf (pre-trained) 2D pose detectors -- no finetuning on the 3D task is even needed. The key observation is that, while the pose detector learns to localize 2D joints, such representations (e.g., feature maps) implicitly encode the joint-centric spatial context thanks to the regional operations in backbone networks. We design a simple baseline named Context-Aware PoseFormer to showcase its effectiveness. Without access to any temporal information, the proposed method significantly outperforms its context-agnostic counterpart, PoseFormer, and other state-of-the-art methods using up to hundreds of video frames regarding both speed and precision. Project page: https://qitaozhao.github.io/ContextAware-PoseFormer