Rapid Adaptation of Neural Machine Translation to New Languages
It addresses rapid adaptation for low-resourced languages, offering incremental improvements over existing multilingual approaches.
This paper tackles the problem of adapting neural machine translation to new, low-resourced languages by proposing methods based on massively multilingual seed models and similar-language regularization, achieving up to 15.5 BLEU without LRL data and a 1.7 BLEU improvement over other methods.
This paper examines the problem of adapting neural machine translation systems to new, low-resourced languages (LRLs) as effectively and rapidly as possible. We propose methods based on starting with massively multilingual "seed models", which can be trained ahead-of-time, and then continuing training on data related to the LRL. We contrast a number of strategies, leading to a novel, simple, yet effective method of "similar-language regularization", where we jointly train on both a LRL of interest and a similar high-resourced language to prevent over-fitting to small LRL data. Experiments demonstrate that massively multilingual models, even without any explicit adaptation, are surprisingly effective, achieving BLEU scores of up to 15.5 with no data from the LRL, and that the proposed similar-language regularization method improves over other adaptation methods by 1.7 BLEU points average over 4 LRL settings. Code to reproduce experiments at https://github.com/neubig/rapid-adaptation