Hierarchical Graph Convolutional Skeleton Transformer for Action Recognition
This work addresses limitations in skeleton-based action recognition, an incremental improvement for computer vision applications.
The paper tackles the problems of neighborhood constraints and entangled spatiotemporal features in graph convolutional networks for skeleton-based action recognition by proposing a Hierarchical Graph Convolutional skeleton Transformer (HGCT), which achieves superior performance and good interpretability.
Graph convolutional networks (GCNs) have emerged as dominant methods for skeleton-based action recognition. However, they still suffer from two problems, namely, neighborhood constraints and entangled spatiotemporal feature representations. Most studies have focused on improving the design of graph topology to solve the first problem but they have yet to fully explore the latter. In this work, we design a disentangled spatiotemporal transformer (DSTT) block to overcome the above limitations of GCNs in three steps: (i) feature disentanglement for spatiotemporal decomposition;(ii) global spatiotemporal attention for capturing correlations in the global context; and (iii) local information enhancement for utilizing more local information. Thereon, we propose a novel architecture, named Hierarchical Graph Convolutional skeleton Transformer (HGCT), to employ the complementary advantages of GCN (i.e., local topology, temporal dynamics and hierarchy) and Transformer (i.e., global context and dynamic attention). HGCT is lightweight and computationally efficient. Quantitative analysis demonstrates the superiority and good interpretability of HGCT.