MoLE : Mixture of Language Experts for Multi-Lingual Automatic Speech Recognition
This work addresses the problem of efficient and accurate speech recognition across multiple languages, especially benefiting low-resource language scenarios, but it appears incremental as it builds on existing language-aware paradigms.
The paper tackles multi-lingual automatic speech recognition by proposing MoLE, a network that uses a mixture of language experts to process speech in various languages, showing improved performance, particularly for low-resource languages, with experimental results in a 5-language scenario.
Multi-lingual speech recognition aims to distinguish linguistic expressions in different languages and integrate acoustic processing simultaneously. In contrast, current multi-lingual speech recognition research follows a language-aware paradigm, mainly targeted to improve recognition performance rather than discriminate language characteristics. In this paper, we present a multi-lingual speech recognition network named Mixture-of-Language-Expert(MoLE), which digests speech in a variety of languages. Specifically, MoLE analyzes linguistic expression from input speech in arbitrary languages, activating a language-specific expert with a lightweight language tokenizer. The tokenizer not only activates experts, but also estimates the reliability of the activation. Based on the reliability, the activated expert and the language-agnostic expert are aggregated to represent language-conditioned embedding for efficient speech recognition. Our proposed model is evaluated in 5 languages scenario, and the experimental results show that our structure is advantageous on multi-lingual recognition, especially for speech in low-resource language.