Semi-Supervised Sound Event Detection with Conditional Mixup and Embedding-Level Contrastive Loss
For researchers in sound event detection, this paper provides an incremental improvement over the existing ATST-SED framework by better leveraging unlabeled data through contrastive learning and a novel mixup technique.
This work improves semi-supervised sound event detection by introducing an embedding-level contrastive loss and a conditional mixup strategy that resolves the mismatch between mixup roles in pseudo-label and contrastive learning. The model achieves state-of-the-art performance with 0.645 PSDS1 and 0.822 PSDS2 on the DESED validation set.
Sound event detection (SED) is a core module for acoustic environmental analysis, yet its performance is often limited by scarce labeled data. Recent systems leverage large pretrained audio foundation models, but effective fine-tuning remains challenging because labeled data are limited while unlabeled data are abundant. A previous work, ATST-SED, addressed this problem with a pseudo-label based semi-supervised fine-tuning framework. In this work, we further improve the framework by adopting an embedding-level self-supervised contrastive loss inspired by ATST-Frame pretraining. This contrastive objective better exploits unlabeled data during fine-tuning. One challenge is that mixup serves different roles in the two objectives: pseudo-label learning uses composition mixup, while contrastive learning treats mixup as a perturbation. To resolve this mismatch, we propose conditional mixup, which combines composition mixup and perturbation mixup in one semi-supervised framework and defines the corresponding embedding-level contrastive losses. The resulting model achieves 0.645 PSDS1 and 0.822 PSDS2 on the DESED validation set, establishing a new state of the art.