Enhancing Code-Switching Speech Recognition with LID-Based Collaborative Mixture of Experts Model
This work addresses the problem of accurate speech recognition in multilingual contexts for applications like voice assistants, though it appears incremental by building on existing MoE frameworks.
The paper tackles the challenge of code-switching speech recognition by proposing a Collaborative-MoE model that uses Language Identification (LID) to route and integrate language-specific experts, achieving significant performance enhancements compared to alternative methods.
Due to the inherent difficulty in modeling phonetic similarities across different languages, code-switching speech recognition presents a formidable challenge. This study proposes a Collaborative-MoE, a Mixture of Experts (MoE) model that leverages a collaborative mechanism among expert groups. Initially, a preceding routing network explicitly learns Language Identification (LID) tasks and selects experts based on acquired LID weights. This process ensures robust routing information to the MoE layer, mitigating interference from diverse language domains on expert network parameter updates. The LID weights are also employed to facilitate inter-group collaboration, enabling the integration of language-specific representations. Furthermore, within each language expert group, a gating network operates unsupervised to foster collaboration on attributes beyond language. Extensive experiments demonstrate the efficacy of our approach, achieving significant performance enhancements compared to alternative methods. Importantly, our method preserves the efficient inference capabilities characteristic of MoE models without necessitating additional pre-training.