Multi Scale Temporal Graph Networks For Skeleton-based Action Recognition
This work offers an incremental improvement in skeleton-based action recognition performance for computer vision researchers by proposing a novel network architecture and graph strategy.
This paper addresses limitations in existing Graph Convolutional Networks (GCNs) for skeleton-based action recognition by proposing Temporal Graph Networks (TGN) to simultaneously extract spatiotemporal features. Additionally, it introduces a multi-scale graph strategy (full-scale, part-scale, core-scale) to better describe joint relationships beyond physical connections. The proposed TGN with this graph strategy outperforms state-of-the-art methods on two large datasets.
Graph convolutional networks (GCNs) can effectively capture the features of related nodes and improve the performance of the model. More attention is paid to employing GCN in Skeleton-Based action recognition. But existing methods based on GCNs have two problems. First, the consistency of temporal and spatial features is ignored for extracting features node by node and frame by frame. To obtain spatiotemporal features simultaneously, we design a generic representation of skeleton sequences for action recognition and propose a novel model called Temporal Graph Networks (TGN). Secondly, the adjacency matrix of the graph describing the relation of joints is mostly dependent on the physical connection between joints. To appropriately describe the relations between joints in the skeleton graph, we propose a multi-scale graph strategy, adopting a full-scale graph, part-scale graph, and core-scale graph to capture the local features of each joint and the contour features of important joints. Experiments were carried out on two large datasets and results show that TGN with our graph strategy outperforms state-of-the-art methods.