Boosting Unsupervised Machine Translation with Pseudo-Parallel Data
This addresses the problem of low-quality machine translation for low-resource languages, but it is incremental as it builds on existing unsupervised methods.
The paper tackled the challenge of improving unsupervised machine translation for low-resource languages by proposing a training strategy that incorporates pseudo-parallel sentence pairs mined from monolingual corpora, achieving an improvement of up to 14.5 BLEU points over a baseline on English to Ukrainian translation.
Even with the latest developments in deep learning and large-scale language modeling, the task of machine translation (MT) of low-resource languages remains a challenge. Neural MT systems can be trained in an unsupervised way without any translation resources but the quality lags behind, especially in truly low-resource conditions. We propose a training strategy that relies on pseudo-parallel sentence pairs mined from monolingual corpora in addition to synthetic sentence pairs back-translated from monolingual corpora. We experiment with different training schedules and reach an improvement of up to 14.5 BLEU points (English to Ukrainian) over a baseline trained on back-translated data only.