Enhancing Synthetic Training Data for Speech Commands: From ASR-Based Filtering to Domain Adaptation in SSL Latent Space
This work addresses data quality issues in speech command classification for researchers and practitioners using synthetic data augmentation, though it is incremental as it builds on existing filtering and adaptation techniques.
The study tackled the problem of low-quality synthetic speech data harming speech command classification by showing that ASR-based filtering improves data quality and performance, achieving a 5% accuracy gain on the Google Speech Commands dataset, and used a CycleGAN to reduce the gap between synthetic and real speech features.
The use of synthetic speech as data augmentation is gaining increasing popularity in fields such as automatic speech recognition and speech classification tasks. Despite novel text-to-speech systems with voice cloning capabilities, that allow the usage of a larger amount of voices based on short audio segments, it is known that these systems tend to hallucinate and oftentimes produce bad data that will most likely have a negative impact on the downstream task. In the present work, we conduct a set of experiments around zero-shot learning with synthetic speech data for the specific task of speech commands classification. Our results on the Google Speech Commands dataset show that a simple ASR-based filtering method can have a big impact in the quality of the generated data, translating to a better performance. Furthermore, despite the good quality of the generated speech data, we also show that synthetic and real speech can still be easily distinguishable when using self-supervised (WavLM) features, an aspect further explored with a CycleGAN to bridge the gap between the two types of speech material.