Discrepancy-Aware Attention Network for Enhanced Audio-Visual Zero-Shot Learning
This work addresses modal imbalance issues in audio-visual zero-shot learning, which is important for improving video classification tasks, but it appears incremental as it builds on existing methods to handle discrepancies.
The paper tackled the problem of modal imbalance in audio-visual zero-shot learning, which reduces discriminative capabilities for unseen classes, and proposed a Discrepancy-Aware Attention Network (DAAN) that achieved state-of-the-art performance on benchmark datasets.
Audio-visual Zero-Shot Learning (ZSL) has attracted significant attention for its ability to identify unseen classes and perform well in video classification tasks. However, modal imbalance in (G)ZSL leads to over-reliance on the optimal modality, reducing discriminative capabilities for unseen classes. Some studies have attempted to address this issue by modifying parameter gradients, but two challenges still remain: (a) Quality discrepancies, where modalities offer differing quantities and qualities of information for the same concept. (b) Content discrepancies, where sample contributions within a modality vary significantly. To address these challenges, we propose a Discrepancy-Aware Attention Network (DAAN) for Enhanced Audio-Visual ZSL. Our approach introduces a Quality-Discrepancy Mitigation Attention (QDMA) unit to minimize redundant information in the high-quality modality and a Contrastive Sample-level Gradient Modulation (CSGM) block to adjust gradient magnitudes and balance content discrepancies. We quantify modality contributions by integrating optimization and convergence rate for more precise gradient modulation in CSGM. Experiments demonstrates DAAN achieves state-of-the-art performance on benchmark datasets, with ablation studies validating the effectiveness of individual modules.