Skeletal Human Action Recognition using Hybrid Attention based Graph Convolutional Network
This work addresses action recognition for applications like surveillance or human-computer interaction, but it is incremental as it builds on existing Graph Convolutional Networks with specific enhancements.
The paper tackled the problem of skeleton-based action recognition by addressing the limitations of existing Graph Convolutional Networks, which ignore dynamic relations between distant joints like head, hands, and feet, and introduced a new adaptive spatial attention layer and initial graph adjacency matrix, resulting in visible improvements in action recognition accuracy on NTU-RGB+D and Kinetics skeleton datasets.
In skeleton-based action recognition, Graph Convolutional Networks model human skeletal joints as vertices and connect them through an adjacency matrix, which can be seen as a local attention mask. However, in most existing Graph Convolutional Networks, the local attention mask is defined based on natural connections of human skeleton joints and ignores the dynamic relations for example between head, hands and feet joints. In addition, the attention mechanism has been proven effective in Natural Language Processing and image description, which is rarely investigated in existing methods. In this work, we proposed a new adaptive spatial attention layer that extends local attention map to global based on relative distance and relative angle information. Moreover, we design a new initial graph adjacency matrix that connects head, hands and feet, which shows visible improvement in terms of action recognition accuracy. The proposed model is evaluated on two large-scale and challenging datasets in the field of human activities in daily life: NTU-RGB+D and Kinetics skeleton. The results demonstrate that our model has strong performance on both dataset.