Language Complexity and Speech Recognition Accuracy: Orthographic Complexity Hurts, Phonological Complexity Doesn't
This addresses the problem of improving ASR performance for linguists and developers by identifying orthographic complexity as a key bottleneck, though it is incremental as it builds on existing multilingual models.
The study investigated how orthographic and phonological complexities affect Automatic Speech Recognition (ASR) accuracy, finding that orthographic complexities significantly correlate with lower accuracy, while phonological complexity does not show a significant correlation.
We investigate what linguistic factors affect the performance of Automatic Speech Recognition (ASR) models. We hypothesize that orthographic and phonological complexities both degrade accuracy. To examine this, we fine-tune the multilingual self-supervised pretrained model Wav2Vec2-XLSR-53 on 25 languages with 15 writing systems, and we compare their ASR accuracy, number of graphemes, unigram grapheme entropy, logographicity (how much word/morpheme-level information is encoded in the writing system), and number of phonemes. The results demonstrate that orthographic complexities significantly correlate with low ASR accuracy, while phonological complexity shows no significant correlation.