Optimizing Transformer for Low-Resource Neural Machine Translation
This work addresses translation quality for low-resource language pairs, but it is incremental as it focuses on hyper-parameter tuning rather than a new method.
The paper tackles the challenge of low-resource neural machine translation by optimizing Transformer hyper-parameters, resulting in an improvement of up to 7.3 BLEU points compared to default settings.
Language pairs with limited amounts of parallel data, also known as low-resource languages, remain a challenge for neural machine translation. While the Transformer model has achieved significant improvements for many language pairs and has become the de facto mainstream architecture, its capability under low-resource conditions has not been fully investigated yet. Our experiments on different subsets of the IWSLT14 training data show that the effectiveness of Transformer under low-resource conditions is highly dependent on the hyper-parameter settings. Our experiments show that using an optimized Transformer for low-resource conditions improves the translation quality up to 7.3 BLEU points compared to using the Transformer default settings.