Optimizing Local-Global Dependencies for Accurate 3D Human Pose Estimation
This work improves 3D human pose estimation for applications like human mesh recovery, but it is incremental as it builds on transformer-based methods by adding local feature extraction.
The paper tackles the problem of 3D human pose estimation by addressing the limitation of transformer-based methods in capturing fine-grained local details, proposing SSR-STF, a dual-stream model that integrates local features with global dependencies. The result is state-of-the-art performance with P1 errors of 37.4 mm on Human3.6M and 13.2 mm on MPI-INF-3DHP, outperforming existing methods in accuracy and generalization.
Transformer-based methods have recently achieved significant success in 3D human pose estimation, owing to their strong ability to model long-range dependencies. However, relying solely on the global attention mechanism is insufficient for capturing the fine-grained local details, which are crucial for accurate pose estimation. To address this, we propose SSR-STF, a dual-stream model that effectively integrates local features with global dependencies to enhance 3D human pose estimation. Specifically, we introduce SSRFormer, a simple yet effective module that employs the skeleton selective refine attention (SSRA) mechanism to capture fine-grained local dependencies in human pose sequences, complementing the global dependencies modeled by the Transformer. By adaptively fusing these two feature streams, SSR-STF can better learn the underlying structure of human poses, overcoming the limitations of traditional methods in local feature extraction. Extensive experiments on the Human3.6M and MPI-INF-3DHP datasets demonstrate that SSR-STF achieves state-of-the-art performance, with P1 errors of 37.4 mm and 13.2 mm respectively, outperforming existing methods in both accuracy and generalization. Furthermore, the motion representations learned by our model prove effective in downstream tasks such as human mesh recovery. Codes are available at https://github.com/poker-xu/SSR-STF.