A Multilingual View of Unsupervised Machine Translation
This work addresses the problem of improving translation quality in low-resource language pairs for NLP applications, representing an incremental advance.
The paper tackles unsupervised machine translation by introducing a probabilistic framework that incorporates auxiliary parallel data via a cross-translation loss, achieving higher BLEU scores, such as a +1.65 advantage in Romanian-English translation.
We present a probabilistic framework for multilingual neural machine translation that encompasses supervised and unsupervised setups, focusing on unsupervised translation. In addition to studying the vanilla case where there is only monolingual data available, we propose a novel setup where one language in the (source, target) pair is not associated with any parallel data, but there may exist auxiliary parallel data that contains the other. This auxiliary data can naturally be utilized in our probabilistic framework via a novel cross-translation loss term. Empirically, we show that our approach results in higher BLEU scores over state-of-the-art unsupervised models on the WMT'14 English-French, WMT'16 English-German, and WMT'16 English-Romanian datasets in most directions. In particular, we obtain a +1.65 BLEU advantage over the best-performing unsupervised model in the Romanian-English direction.