Improving Multilingual ASR in the Wild Using Simple N-best Re-ranking
This addresses the challenge of imperfect spoken language identification impacting ASR accuracy for practical multilingual applications, though it is incremental as it builds on existing acoustic models with a re-ranking technique.
The paper tackled the problem of multilingual automatic speech recognition (ASR) in real-world settings where the spoken language is unknown, by proposing a simple N-best re-ranking method using external features like language models and text-based language identification. The results showed improvements in spoken language identification accuracy by 8.7% and 6.1% for MMS and Whisper models on FLEURS, and reduced word error rates by 3.3% and 2.0%.
Multilingual Automatic Speech Recognition (ASR) models are typically evaluated in a setting where the ground-truth language of the speech utterance is known, however, this is often not the case for most practical settings. Automatic Spoken Language Identification (SLID) models are not perfect and misclassifications have a substantial impact on the final ASR accuracy. In this paper, we present a simple and effective N-best re-ranking approach to improve multilingual ASR accuracy for several prominent acoustic models by employing external features such as language models and text-based language identification models. Our results on FLEURS using the MMS and Whisper models show spoken language identification accuracy improvements of 8.7% and 6.1%, respectively and word error rates which are 3.3% and 2.0% lower on these benchmarks.