Massively Multilingual Adversarial Speech Recognition
This work addresses the challenge of building robust speech recognition systems for diverse languages, though it appears incremental by building on existing multilingual models.
The study investigated adapting multilingual speech recognition models across up to 100 languages, finding that phonetic, phonological, and other linguistic similarities influence performance, and demonstrated that combining a phoneme objective with a language-adversarial objective improves language-independent representations.
We report on adaptation of multilingual end-to-end speech recognition models trained on as many as 100 languages. Our findings shed light on the relative importance of similarity between the target and pretraining languages along the dimensions of phonetics, phonology, language family, geographical location, and orthography. In this context, experiments demonstrate the effectiveness of two additional pretraining objectives in encouraging language-independent encoder representations: a context-independent phoneme objective paired with a language-adversarial classification objective.