English Accent Accuracy Analysis in a State-of-the-Art Automatic Speech Recognition System
This work addresses performance biases in speech systems for underrepresented English accents, which is an incremental but important issue for improving fairness in ASR technologies.
The study evaluated a state-of-the-art automatic speech recognition model on unseen data from diverse English accents, revealing accuracy biases that favor accents more prevalent in the training corpus, with results presented graphically in terms of Word Error Rate.
Nowadays, research in speech technologies has gotten a lot out thanks to recently created public domain corpora that contain thousands of recording hours. These large amounts of data are very helpful for training the new complex models based on deep learning technologies. However, the lack of dialectal diversity in a corpus is known to cause performance biases in speech systems, mainly for underrepresented dialects. In this work, we propose to evaluate a state-of-the-art automatic speech recognition (ASR) deep learning-based model, using unseen data from a corpus with a wide variety of labeled English accents from different countries around the world. The model has been trained with 44.5K hours of English speech from an open access corpus called Multilingual LibriSpeech, showing remarkable results in popular benchmarks. We test the accuracy of such ASR against samples extracted from another public corpus that is continuously growing, the Common Voice dataset. Then, we present graphically the accuracy in terms of Word Error Rate of each of the different English included accents, showing that there is indeed an accuracy bias in terms of accentual variety, favoring the accents most prevalent in the training corpus.