Two-stream joint matching method based on contrastive learning for few-shot action recognition
This work addresses video matching issues in few-shot action recognition, but it appears incremental as it builds on existing metric learning paradigms.
The paper tackles the problem of few-shot action recognition by addressing inadequate action relation modeling and video matching challenges like different lengths and misalignment, proposing a Two-Stream Joint Matching method that achieves effectiveness on SSv2 and Kinetics datasets.
Although few-shot action recognition based on metric learning paradigm has achieved significant success, it fails to address the following issues: (1) inadequate action relation modeling and underutilization of multi-modal information; (2) challenges in handling video matching problems with different lengths and speeds, and video matching problems with misalignment of video sub-actions. To address these issues, we propose a Two-Stream Joint Matching method based on contrastive learning (TSJM), which consists of two modules: Multi-modal Contrastive Learning Module (MCL) and Joint Matching Module (JMM). The objective of the MCL is to extensively investigate the inter-modal mutual information relationships, thereby thoroughly extracting modal information to enhance the modeling of action relationships. The JMM aims to simultaneously address the aforementioned video matching problems. The effectiveness of the proposed method is evaluated on two widely used few shot action recognition datasets, namely, SSv2 and Kinetics. Comprehensive ablation experiments are also conducted to substantiate the efficacy of our proposed approach.