Improving Noise Robustness of Contrastive Speech Representation Learning with Speech Reconstruction
This addresses noise interference in ASR systems for real-world deployment, representing an incremental improvement over existing methods.
The paper tackles noise robustness in automatic speech recognition by combining a reconstruction module with contrastive learning for self-supervised pre-training on noisy data, resulting in a 4.1/7.5% WER reduction on noisy test sets and state-of-the-art performance on CHiME-4 without a denoising front-end.
Noise robustness is essential for deploying automatic speech recognition (ASR) systems in real-world environments. One way to reduce the effect of noise interference is to employ a preprocessing module that conducts speech enhancement, and then feed the enhanced speech to an ASR backend. In this work, instead of suppressing background noise with a conventional cascaded pipeline, we employ a noise-robust representation learned by a refined self-supervised framework for noisy speech recognition. We propose to combine a reconstruction module with contrastive learning and perform multi-task continual pre-training on noisy data. The reconstruction module is used for auxiliary learning to improve the noise robustness of the learned representation and thus is not required during inference. Experiments demonstrate the effectiveness of our proposed method. Our model substantially reduces the word error rate (WER) for the synthesized noisy LibriSpeech test sets, and yields around 4.1/7.5% WER reduction on noisy clean/other test sets compared to data augmentation. For the real-world noisy speech from the CHiME-4 challenge (1-channel track), we have obtained the state of the art ASR performance without any denoising front-end. Moreover, we achieve comparable performance to the best supervised approach reported with only 16% of labeled data.