Edinburgh Neural Machine Translation Systems for WMT 16
This work addresses translation quality for multiple language pairs in a competitive benchmark, though it is incremental as it builds on existing attentional encoder-decoder methods.
The researchers tackled neural machine translation for multiple language pairs in the WMT 2016 task, achieving improvements of 4.3-11.2 BLEU over baselines and ranking as the best constrained system in 7 out of 8 directions in human evaluation.
We participated in the WMT 2016 shared news translation task by building neural translation systems for four language pairs, each trained in both directions: English<->Czech, English<->German, English<->Romanian and English<->Russian. Our systems are based on an attentional encoder-decoder, using BPE subword segmentation for open-vocabulary translation with a fixed vocabulary. We experimented with using automatic back-translations of the monolingual News corpus as additional training data, pervasive dropout, and target-bidirectional models. All reported methods give substantial improvements, and we see improvements of 4.3--11.2 BLEU over our baseline systems. In the human evaluation, our systems were the (tied) best constrained system for 7 out of 8 translation directions in which we participated.