DG-STGCN: Dynamic Spatial-Temporal Modeling for Skeleton-based Action Recognition
This addresses the problem of capturing complex joint correlations in action recognition for computer vision applications, representing an incremental improvement over existing GCN-based methods.
The paper tackles the limitation of fixed graphical structures in graph convolution networks for skeleton-based action recognition by proposing DG-STGCN, which uses learned affinity matrices and adaptive temporal modeling, achieving state-of-the-art performance on benchmarks like NTURGB+D and Kinetics-Skeleton with notable margins.
Graph convolution networks (GCN) have been widely used in skeleton-based action recognition. We note that existing GCN-based approaches primarily rely on prescribed graphical structures (ie., a manually defined topology of skeleton joints), which limits their flexibility to capture complicated correlations between joints. To move beyond this limitation, we propose a new framework for skeleton-based action recognition, namely Dynamic Group Spatio-Temporal GCN (DG-STGCN). It consists of two modules, DG-GCN and DG-TCN, respectively, for spatial and temporal modeling. In particular, DG-GCN uses learned affinity matrices to capture dynamic graphical structures instead of relying on a prescribed one, while DG-TCN performs group-wise temporal convolutions with varying receptive fields and incorporates a dynamic joint-skeleton fusion module for adaptive multi-level temporal modeling. On a wide range of benchmarks, including NTURGB+D, Kinetics-Skeleton, BABEL, and Toyota SmartHome, DG-STGCN consistently outperforms state-of-the-art methods, often by a notable margin.