Double-chain Constraints for 3D Human Pose Estimation in Images and Videos
This addresses the challenge of 3D human pose estimation from images and videos, which is important for applications like animation and human-computer interaction, but appears incremental as it builds on existing GCN and Transformer methods.
The paper tackles the problem of reconstructing 3D human poses from 2D poses by proposing a Double-chain Graph Convolutional Transformer (DC-GCT) model that uses local-to-global and global-to-local chains to capture joint dependencies, achieving state-of-the-art performance on Human3.6M and MPI-INF-3DHP datasets.
Reconstructing 3D poses from 2D poses lacking depth information is particularly challenging due to the complexity and diversity of human motion. The key is to effectively model the spatial constraints between joints to leverage their inherent dependencies. Thus, we propose a novel model, called Double-chain Graph Convolutional Transformer (DC-GCT), to constrain the pose through a double-chain design consisting of local-to-global and global-to-local chains to obtain a complex representation more suitable for the current human pose. Specifically, we combine the advantages of GCN and Transformer and design a Local Constraint Module (LCM) based on GCN and a Global Constraint Module (GCM) based on self-attention mechanism as well as a Feature Interaction Module (FIM). The proposed method fully captures the multi-level dependencies between human body joints to optimize the modeling capability of the model. Moreover, we propose a method to use temporal information into the single-frame model by guiding the video sequence embedding through the joint embedding of the target frame, with negligible increase in computational cost. Experimental results demonstrate that DC-GCT achieves state-of-the-art performance on two challenging datasets (Human3.6M and MPI-INF-3DHP). Notably, our model achieves state-of-the-art performance on all action categories in the Human3.6M dataset using detected 2D poses from CPN, and our code is available at: https://github.com/KHB1698/DC-GCT.