3D Human Pose Estimation with Spatial and Temporal Transformers
This addresses the problem of accurate 3D human pose estimation for computer vision applications, representing an incremental advance by applying transformers to a domain previously dominated by convolutional architectures.
The paper tackles 3D human pose estimation in videos by introducing PoseFormer, a purely transformer-based approach that models spatial and temporal relations, achieving state-of-the-art performance on Human3.6M and MPI-INF-3DHP datasets.
Transformer architectures have become the model of choice in natural language processing and are now being introduced into computer vision tasks such as image classification, object detection, and semantic segmentation. However, in the field of human pose estimation, convolutional architectures still remain dominant. In this work, we present PoseFormer, a purely transformer-based approach for 3D human pose estimation in videos without convolutional architectures involved. Inspired by recent developments in vision transformers, we design a spatial-temporal transformer structure to comprehensively model the human joint relations within each frame as well as the temporal correlations across frames, then output an accurate 3D human pose of the center frame. We quantitatively and qualitatively evaluate our method on two popular and standard benchmark datasets: Human3.6M and MPI-INF-3DHP. Extensive experiments show that PoseFormer achieves state-of-the-art performance on both datasets. Code is available at \url{https://github.com/zczcwh/PoseFormer}