A New Adjacency Matrix Configuration in GCN-based Models for Skeleton-based Action Recognition
This work addresses a fundamental issue in skeleton-based action recognition for computer vision applications, but it is incremental as it builds on existing GCN methods by modifying the adjacency matrix.
The paper tackled the problem of using the human natural skeleton structure adjacency matrix in GCN-based models for skeleton-based action recognition, proposing a new adjacency matrix that adaptively learns joint relationships, which improved model performance, noise robustness, and transferability on datasets like NTURGBD60 and FineGYM.
Human skeleton data has received increasing attention in action recognition due to its background robustness and high efficiency. In skeleton-based action recognition, graph convolutional network (GCN) has become the mainstream method. This paper analyzes the fundamental factor for GCN-based models -- the adjacency matrix. We notice that most GCN-based methods conduct their adjacency matrix based on the human natural skeleton structure. Based on our former work and analysis, we propose that the human natural skeleton structure adjacency matrix is not proper for skeleton-based action recognition. We propose a new adjacency matrix that abandons all rigid neighbor connections but lets the model adaptively learn the relationships of joints. We conduct extensive experiments and analysis with a validation model on two skeleton-based action recognition datasets (NTURGBD60 and FineGYM). Comprehensive experimental results and analysis reveals that 1) the most widely used human natural skeleton structure adjacency matrix is unsuitable in skeleton-based action recognition; 2) The proposed adjacency matrix is superior in model performance, noise robustness and transferability.