WeChat Neural Machine Translation Systems for WMT21
This work addresses machine translation challenges for news domains, presenting incremental improvements in performance for specific language pairs.
The paper tackles the WMT 2021 shared news translation task for multiple language pairs by developing Transformer-based systems with variants, achieving the highest BLEU scores among submissions for English->Chinese (36.9), English->Japanese (46.9), and Japanese->English (27.8), and the highest among constrained submissions for English->German (31.3).
This paper introduces WeChat AI's participation in WMT 2021 shared news translation task on English->Chinese, English->Japanese, Japanese->English and English->German. Our systems are based on the Transformer (Vaswani et al., 2017) with several novel and effective variants. In our experiments, we employ data filtering, large-scale synthetic data generation (i.e., back-translation, knowledge distillation, forward-translation, iterative in-domain knowledge transfer), advanced finetuning approaches, and boosted Self-BLEU based model ensemble. Our constrained systems achieve 36.9, 46.9, 27.8 and 31.3 case-sensitive BLEU scores on English->Chinese, English->Japanese, Japanese->English and English->German, respectively. The BLEU scores of English->Chinese, English->Japanese and Japanese->English are the highest among all submissions, and that of English->German is the highest among all constrained submissions.