Adding Robust Code-Switching Capabilities to High Performance Multilingual ASR
For developers of multilingual ASR systems, it provides a method to add code-switching robustness without sacrificing existing performance.
The paper tackles code-switching in multilingual ASR without degrading monolingual performance. Bayesian factorized adaptation reduces transcription errors on code-switched words by 32.87% and overall WER by 5.31% while preserving baseline capabilities.
Code-switching (CSW) remains challenging for large multi-lingual ASR systems in real-world deployment. While fine-tuning on synthetic CSW data is possible, it generally degrades strong monolingual baselines. Our goal is to preserve these capabilities while extending models to handle complex code-switching, including morphological variations across languages. We propose Bayesian factorized adaptation, which learns to efficiently integrate switching-relevant knowledge into strong pretrained models without overwriting existing capabilities. Requiring only a small amount of synthetic data, our approach reduces transcription errors by 32.87% on code-switched words while improving overall WER by 5.31%, all while maintaining mono-lingual performance. Our results demonstrate that effective CSW adaptation depends more on knowledge integration than data complexity.