Regularizing Learnable Feature Extraction for Automatic Speech Recognition
This work addresses overfitting in neural front-ends for ASR, which is an incremental improvement for speech recognition systems.
The paper tackled the problem of learnable feature extraction front-ends in automatic speech recognition (ASR) underperforming due to overfitting, and it proposed regularization methods including audio perturbations and STFT-domain masking to close the performance gap, achieving effective results.
Neural front-ends are an appealing alternative to traditional, fixed feature extraction pipelines for automatic speech recognition (ASR) systems since they can be directly trained to fit the acoustic model. However, their performance often falls short compared to classical methods, which we show is largely due to their increased susceptibility to overfitting. This work therefore investigates regularization methods for training ASR models with learnable feature extraction front-ends. First, we examine audio perturbation methods and show that larger relative improvements can be obtained for learnable features. Additionally, we identify two limitations in the standard use of SpecAugment for these front-ends and propose masking in the short time Fourier transform (STFT)-domain as a simple but effective modification to address these challenges. Finally, integrating both regularization approaches effectively closes the performance gap between traditional and learnable features.