IIP-Transformer: Intra-Inter-Part Transformer for Skeleton-Based Action Recognition
This work addresses efficiency and robustness issues in skeleton-based action recognition, which is important for applications like human-computer interaction and surveillance, but it is incremental as it builds on existing Transformer approaches.
The paper tackled the problem of high computational cost and noise sensitivity in Transformer-based skeleton action recognition by proposing IIP-Transformer, which captures joint-level and part-level dependencies, achieving state-of-the-art performance on NTU-RGB+D datasets with over 8x less computational complexity than the previous SOTA method.
Recently, Transformer-based networks have shown great promise on skeleton-based action recognition tasks. The ability to capture global and local dependencies is the key to success while it also brings quadratic computation and memory cost. Another problem is that previous studies mainly focus on the relationships among individual joints, which often suffers from the noisy skeleton joints introduced by the noisy inputs of sensors or inaccurate estimations. To address the above issues, we propose a novel Transformer-based network (IIP-Transformer). Instead of exploiting interactions among individual joints, our IIP-Transformer incorporates body joints and parts interactions simultaneously and thus can capture both joint-level (intra-part) and part-level (inter-part) dependencies efficiently and effectively. From the data aspect, we introduce a part-level skeleton data encoding that significantly reduces the computational complexity and is more robust to joint-level skeleton noise. Besides, a new part-level data augmentation is proposed to improve the performance of the model. On two large-scale datasets, NTU-RGB+D 60 and NTU RGB+D 120, the proposed IIP-Transformer achieves the-state-of-art performance with more than 8x less computational complexity than DSTA-Net, which is the SOTA Transformer-based method.