Whilter: A Whisper-based Data Filter for "In-the-Wild" Speech Corpora Using Utterance-level Multi-Task Classification
This work addresses a domain-specific problem for researchers and practitioners using in-the-wild speech corpora, offering an incremental improvement in filtering efficiency and accuracy.
The paper tackles the problem of filtering undesirable features like multiple speakers and non-target languages from large-scale in-the-wild speech datasets, proposing Whilter, a multitask model that achieves F1 scores above 85% and equal error rates of 6.5% to 7.8% for three subtasks, outperforming a state-of-the-art BEATs classifier with reduced processing time.
Large-scale in-the-wild speech datasets have become more prevalent in recent years due to increased interest in models that can learn useful features from unlabelled data for tasks such as speech recognition or synthesis. These datasets often contain undesirable features, such as multiple speakers, non-target languages, and music, which may impact model learning. The Whilter model is proposed as a multitask solution to identify these undesirable samples. Whilter uses a Whisper encoder with an attention-based classifier to solve five diverse classification problems at once. In addition, an annotated dataset is published for a subset of two popular in-the-wild corpora. Whilter achieves F1 scores above 85% and equal error rates of 6.5% to 7.8% for three of five subtasks, outperforming a state-of-the-art BEATs classifier on speech-specific classes, with a notable decrease in processing time compared to a combination of single-task alternatives.