HLT-MT: High-resource Language-specific Training for Multilingual Neural Machine Translation
This addresses translation quality degradation in multilingual models, offering a solution for high-resource language pairs, though it is incremental as it builds on existing multilingual training approaches.
The paper tackles the problem of language interference in multilingual neural machine translation, particularly for high-resource languages, by proposing a two-stage training method with language-specific modules, resulting in improved performance on benchmarks like WMT-10 and OPUS-100.
Multilingual neural machine translation (MNMT) trained in multiple language pairs has attracted considerable attention due to fewer model parameters and lower training costs by sharing knowledge among multiple languages. Nonetheless, multilingual training is plagued by language interference degeneration in shared parameters because of the negative interference among different translation directions, especially on high-resource languages. In this paper, we propose the multilingual translation model with the high-resource language-specific training (HLT-MT) to alleviate the negative interference, which adopts the two-stage training with the language-specific selection mechanism. Specifically, we first train the multilingual model only with the high-resource pairs and select the language-specific modules at the top of the decoder to enhance the translation quality of high-resource directions. Next, the model is further trained on all available corpora to transfer knowledge from high-resource languages (HRLs) to low-resource languages (LRLs). Experimental results show that HLT-MT outperforms various strong baselines on WMT-10 and OPUS-100 benchmarks. Furthermore, the analytic experiments validate the effectiveness of our method in mitigating the negative interference in multilingual training.