Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages
This work addresses the problem of low-resource language translation for NLP practitioners, showing significant improvements but being incremental in its approach.
The paper tackles the poor performance of unsupervised machine translation for rare languages, which often achieves less than 3.0 BLEU, by introducing a multilingual model that leverages auxiliary data and a three-stage training scheme, resulting in gains of up to 14.4 BLEU and outperforming state-of-the-art unsupervised and some supervised baselines.
Unsupervised translation has reached impressive performance on resource-rich language pairs such as English-French and English-German. However, early studies have shown that in more realistic settings involving low-resource, rare languages, unsupervised translation performs poorly, achieving less than 3.0 BLEU. In this work, we show that multilinguality is critical to making unsupervised systems practical for low-resource settings. In particular, we present a single model for 5 low-resource languages (Gujarati, Kazakh, Nepali, Sinhala, and Turkish) to and from English directions, which leverages monolingual and auxiliary parallel data from other high-resource language pairs via a three-stage training scheme. We outperform all current state-of-the-art unsupervised baselines for these languages, achieving gains of up to 14.4 BLEU. Additionally, we outperform a large collection of supervised WMT submissions for various language pairs as well as match the performance of the current state-of-the-art supervised model for Nepali-English. We conduct a series of ablation studies to establish the robustness of our model under different degrees of data quality, as well as to analyze the factors which led to the superior performance of the proposed approach over traditional unsupervised models.