Meta Back-translation
This addresses a key bottleneck in neural machine translation by optimizing pseudo-parallel data generation for improved model performance, though it is an incremental advancement over existing back-translation techniques.
The paper tackles the problem that higher-quality pseudo-parallel data from back-translation does not always improve neural machine translation models, proposing a meta-learning method to adapt a pre-trained back-translation model to generate data that better trains forward-translation models, resulting in significant improvements on WMT En-De'14, WMT En-Fr'14, and multilingual settings.
Back-translation is an effective strategy to improve the performance of Neural Machine Translation~(NMT) by generating pseudo-parallel data. However, several recent works have found that better translation quality of the pseudo-parallel data does not necessarily lead to better final translation models, while lower-quality but more diverse data often yields stronger results. In this paper, we propose a novel method to generate pseudo-parallel data from a pre-trained back-translation model. Our method is a meta-learning algorithm which adapts a pre-trained back-translation model so that the pseudo-parallel data it generates would train a forward-translation model to do well on a validation set. In our evaluations in both the standard datasets WMT En-De'14 and WMT En-Fr'14, as well as a multilingual translation setting, our method leads to significant improvements over strong baselines. Our code will be made available.