Topology-aware Convolutional Neural Network for Efficient Skeleton-based Action Recognition
This work addresses the challenge of efficient action recognition from skeleton data for applications in human-computer interaction and surveillance, offering a novel CNN-based approach that is incremental in improving upon existing methods.
The paper tackles the problem of skeleton-based action recognition by proposing a pure CNN architecture called Topology-aware CNN (Ta-CNN), which effectively models irregular skeleton topology through a cross-channel feature augmentation module and a SkeletonMix strategy, achieving comparable performance to leading GCN-based methods with significantly lower complexity in terms of GFLOPs and parameters on datasets like NTU RGB+D 120.
In the context of skeleton-based action recognition, graph convolutional networks (GCNs) have been rapidly developed, whereas convolutional neural networks (CNNs) have received less attention. One reason is that CNNs are considered poor in modeling the irregular skeleton topology. To alleviate this limitation, we propose a pure CNN architecture named Topology-aware CNN (Ta-CNN) in this paper. In particular, we develop a novel cross-channel feature augmentation module, which is a combo of map-attend-group-map operations. By applying the module to the coordinate level and the joint level subsequently, the topology feature is effectively enhanced. Notably, we theoretically prove that graph convolution is a special case of normal convolution when the joint dimension is treated as channels. This confirms that the topology modeling power of GCNs can also be implemented by using a CNN. Moreover, we creatively design a SkeletonMix strategy which mixes two persons in a unique manner and further boosts the performance. Extensive experiments are conducted on four widely used datasets, i.e. N-UCLA, SBU, NTU RGB+D and NTU RGB+D 120 to verify the effectiveness of Ta-CNN. We surpass existing CNN-based methods significantly. Compared with leading GCN-based methods, we achieve comparable performance with much less complexity in terms of the required GFLOPs and parameters.