Automatic Data Augmentation for Domain Adapted Fine-Tuning of Self-Supervised Speech Representations
This addresses domain adaptation for speech recognition in low-resource settings, but it is incremental as it builds on existing SSL and data augmentation techniques.
The paper tackles the problem of acoustic mismatch between pretraining and target datasets in self-supervised speech representations by proposing a supervised domain adaptation method that uses automatically selected data augmentations on a clean dataset to improve fine-tuning. The method was validated in controlled and low-resource domains, achieving better performance compared to baselines.
Self-Supervised Learning (SSL) has allowed leveraging large amounts of unlabeled speech data to improve the performance of speech recognition models even with small annotated datasets. Despite this, speech SSL representations may fail while facing an acoustic mismatch between the pretraining and target datasets. To address this issue, we propose a novel supervised domain adaptation method, designed for cases exhibiting such a mismatch in acoustic domains. It consists in applying properly calibrated data augmentations on a large clean dataset, bringing it closer to the target domain, and using it as part of an initial fine-tuning stage. Augmentations are automatically selected through the minimization of a conditional-dependence estimator, based on the target dataset. The approach is validated during an oracle experiment with controlled distortions and on two amateur-collected low-resource domains, reaching better performances compared to the baselines in both cases.