A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translation
This work addresses optimization challenges in multilingual translation for low-resource languages, though it is incremental as it builds on existing subword methods.
The paper systematically studied how subword segmentation affects cross-lingual transfer in multilingual machine translation, finding that subword regularization boosts synergy in modeling and BPE aids transfer during fine-tuning, with orthographic word boundary differences posing a major barrier.
Multilingual modelling can improve machine translation for low-resource languages, partly through shared subword representations. This paper studies the role of subword segmentation in cross-lingual transfer. We systematically compare the efficacy of several subword methods in promoting synergy and preventing interference across different linguistic typologies. Our findings show that subword regularisation boosts synergy in multilingual modelling, whereas BPE more effectively facilitates transfer during cross-lingual fine-tuning. Notably, our results suggest that differences in orthographic word boundary conventions (the morphological granularity of written words) may impede cross-lingual transfer more significantly than linguistic unrelatedness. Our study confirms that decisions around subword modelling can be key to optimising the benefits of multilingual modelling.