Improving the Training Recipe for a Robust Conformer-based Hybrid Model
This work addresses speaker adaptation for ASR systems, building incrementally on prior methods.
The paper tackles speaker adaptation for robust automatic speech recognition by proposing a Weighted-Simple-Add method for speaker adaptive training, achieving 3.5% and 4.5% relative WER improvements on specific datasets, and extends a previous training recipe to achieve an 11% relative WER improvement with a 34% parameter reduction.
Speaker adaptation is important to build robust automatic speech recognition (ASR) systems. In this work, we investigate various methods for speaker adaptive training (SAT) based on feature-space approaches for a conformer-based acoustic model (AM) on the Switchboard 300h dataset. We propose a method, called Weighted-Simple-Add, which adds weighted speaker information vectors to the input of the multi-head self-attention module of the conformer AM. Using this method for SAT, we achieve 3.5% and 4.5% relative improvement in terms of WER on the CallHome part of Hub5'00 and Hub5'01 respectively. Moreover, we build on top of our previous work where we proposed a novel and competitive training recipe for a conformer-based hybrid AM. We extend and improve this recipe where we achieve 11% relative improvement in terms of word-error-rate (WER) on Switchboard 300h Hub5'00 dataset. We also make this recipe efficient by reducing the total number of parameters by 34% relative.