Robust Automatic Speech Recognition via WavAugment Guided Phoneme Adversarial Training
This work addresses robustness in ASR for practical applications, though it appears incremental as it builds on existing adversarial training and augmentation techniques.
The paper tackles the challenge of making automatic speech recognition (ASR) robust to small perturbations and domain shifts while maintaining clean sample performance, proposing a method that achieves a 6.28% reduction in word error rate on a benchmark, setting a new state-of-the-art.
Developing a practically-robust automatic speech recognition (ASR) is challenging since the model should not only maintain the original performance on clean samples, but also achieve consistent efficacy under small volume perturbations and large domain shifts. To address this problem, we propose a novel WavAugment Guided Phoneme Adversarial Training (wapat). wapat use adversarial examples in phoneme space as augmentation to make the model invariant to minor fluctuations in phoneme representation and preserve the performance on clean samples. In addition, wapat utilizes the phoneme representation of augmented samples to guide the generation of adversaries, which helps to find more stable and diverse gradient-directions, resulting in improved generalization. Extensive experiments demonstrate the effectiveness of wapat on End-to-end Speech Challenge Benchmark (ESB). Notably, SpeechLM-wapat outperforms the original model by 6.28% WER reduction on ESB, achieving the new state-of-the-art.