Affinity Contrastive Learning for Skeleton-based Human Activity Understanding
This work addresses the challenge of improving feature discrimination for skeleton-based human activity understanding, which is important for applications like action recognition and gait analysis, but it is incremental as it builds on existing contrastive learning methods.
The authors tackled the problem of insufficient exploitation of structural inter-class similarities and anomalous positive samples in skeleton-based human activity understanding by introducing ACLNet, which uses an affinity metric and dynamic temperature schedule to improve feature discrimination, achieving state-of-the-art results on multiple datasets like NTU RGB+D 60 and Kinetics-Skeleton.
In skeleton-based human activity understanding, existing methods often adopt the contrastive learning paradigm to construct a discriminative feature space. However, many of these approaches fail to exploit the structural inter-class similarities and overlook the impact of anomalous positive samples. In this study, we introduce ACLNet, an Affinity Contrastive Learning Network that explores the intricate clustering relationships among human activity classes to improve feature discrimination. Specifically, we propose an affinity metric to refine similarity measurements, thereby forming activity superclasses that provide more informative contrastive signals. A dynamic temperature schedule is also introduced to adaptively adjust the penalty strength for various superclasses. In addition, we employ a margin-based contrastive strategy to improve the separation of hard positive and negative samples within classes. Extensive experiments on NTU RGB+D 60, NTU RGB+D 120, Kinetics-Skeleton, PKU-MMD, FineGYM, and CASIA-B demonstrate the superiority of our method in skeleton-based action recognition, gait recognition, and person re-identification. The source code is available at https://github.com/firework8/ACLNet.