Late fusion ensembles for speech recognition on diverse input audio representations
This work addresses incremental improvements in speech recognition accuracy for applications using ensemble methods, though it is incremental as it builds on existing techniques.
The paper tackles the problem of improving automatic speech recognition performance by exploring late fusion ensembles of E-Branchformer models trained on diverse speech audio representations, achieving improvements of 1% to 14% over state-of-the-art models on benchmark datasets like Librispeech and Aishell.
We explore diverse representations of speech audio, and their effect on a performance of late fusion ensemble of E-Branchformer models, applied to Automatic Speech Recognition (ASR) task. Although it is generally known that ensemble methods often improve the performance of the system even for speech recognition, it is very interesting to explore how ensembles of complex state-of-the-art models, such as medium-sized and large E-Branchformers, cope in this setting when their base models are trained on diverse representations of the input speech audio. The results are evaluated on four widely-used benchmark datasets: \textit{Librispeech, Aishell, Gigaspeech}, \textit{TEDLIUMv2} and show that improvements of $1\% - 14\%$ can still be achieved over the state-of-the-art models trained using comparable techniques on these datasets. A noteworthy observation is that such ensemble offers improvements even with the use of language models, although the gap is closing.