Seamless Language Expansion: Enhancing Multilingual Mastery in Self-Supervised Models
This addresses the costly challenge of language expansion in self-supervised models for real-world multilingual applications, though it is incremental.
The paper tackles the problem of efficiently adapting self-supervised models to new languages without degrading performance on existing ones, achieving a MOS increase of about 1.6 and a WER reduction of up to 61.72% for Mandarin speech re-synthesis.
Self-supervised (SSL) models have shown great performance in various downstream tasks. However, they are typically developed for limited languages, and may encounter new languages in real-world. Developing a SSL model for each new language is costly. Thus, it is vital to figure out how to efficiently adapt existed SSL models to a new language without impairing its original abilities. We propose adaptation methods which integrate LoRA to existed SSL models to extend new language. We also develop preservation strategies which include data combination and re-clustering to retain abilities on existed languages. Applied to mHuBERT, we investigate their effectiveness on speech re-synthesis task. Experiments show that our adaptation methods enable mHuBERT to be applied to a new language (Mandarin) with MOS value increased about 1.6 and the relative value of WER reduced up to 61.72%. Also, our preservation strategies ensure that the performance on both existed and new languages remains intact.