Trivial Transfer Learning for Low-Resource Neural Machine Translation
This work addresses the challenge of improving translation quality for low-resource languages, which is crucial for speakers of those languages, but it is incremental as it builds on existing transfer learning concepts with a simplified approach.
The paper tackles the problem of low-resource neural machine translation by proposing a simple transfer learning method where a model trained on a high-resource language pair is fine-tuned on a low-resource pair, resulting in significant improvements over baseline models trained only on low-resource data, even for unrelated languages with different alphabets.
Transfer learning has been proven as an effective technique for neural machine translation under low-resource conditions. Existing methods require a common target language, language relatedness, or specific training tricks and regimes. We present a simple transfer learning method, where we first train a "parent" model for a high-resource language pair and then continue the training on a lowresource pair only by replacing the training corpus. This "child" model performs significantly better than the baseline trained for lowresource pair only. We are the first to show this for targeting different languages, and we observe the improvements even for unrelated languages with different alphabets.