Parts-Mamba: Augmenting Joint Context with Part-Level Scanning for Occluded Human Skeleton
This addresses occluded skeleton recognition for action analysis, with incremental improvements over existing GCN models.
The paper tackles the problem of skeleton action recognition under occlusion by proposing Parts-Mamba, a hybrid GCN-Mamba model that captures part-specific and distant joint context, achieving up to 12.9% improvement in accuracy on NTU RGB+D datasets.
Skeleton action recognition involves recognizing human action from human skeletons. The use of graph convolutional networks (GCNs) has driven major advances in this recognition task. In real-world scenarios, the captured skeletons are not always perfect or complete because of occlusions of parts of the human body or poor communication quality, leading to missing parts in skeletons or videos with missing frames. In the presence of such non-idealities, existing GCN models perform poorly due to missing local context. To address this limitation, we propose Parts-Mamba, a hybrid GCN-Mamba model designed to enhance the ability to capture and maintain contextual information from distant joints. The proposed Parts-Mamba model effectively captures part-specific information through its parts-specific scanning feature and preserves non-neighboring joint context via a parts-body fusion module. Our proposed model is evaluated on the NTU RGB+D 60 and NTU RGB+D 120 datasets under different occlusion settings, achieving up to 12.9% improvement in accuracy.