GaitMA: Pose-guided Multi-modal Feature Fusion for Gait Recognition
This work addresses gait recognition for biometric identification, offering an incremental improvement by combining existing modalities with novel fusion techniques.
The authors tackled the problem of gait recognition by proposing GaitMA, a framework that fuses silhouette and skeleton modalities to overcome limitations like occlusions and lack of dense features, achieving state-of-the-art results on datasets such as Gait3D, OU-MVLP, and CASIA-B.
Gait recognition is a biometric technology that recognizes the identity of humans through their walking patterns. Existing appearance-based methods utilize CNN or Transformer to extract spatial and temporal features from silhouettes, while model-based methods employ GCN to focus on the special topological structure of skeleton points. However, the quality of silhouettes is limited by complex occlusions, and skeletons lack dense semantic features of the human body. To tackle these problems, we propose a novel gait recognition framework, dubbed Gait Multi-model Aggregation Network (GaitMA), which effectively combines two modalities to obtain a more robust and comprehensive gait representation for recognition. First, skeletons are represented by joint/limb-based heatmaps, and features from silhouettes and skeletons are respectively extracted using two CNN-based feature extractors. Second, a co-attention alignment module is proposed to align the features by element-wise attention. Finally, we propose a mutual learning module, which achieves feature fusion through cross-attention, Wasserstein loss is further introduced to ensure the effective fusion of two modalities. Extensive experimental results demonstrate the superiority of our model on Gait3D, OU-MVLP, and CASIA-B.