Contrastive Learning for Many-to-many Multilingual Neural Machine Translation
This work addresses the gap in translation quality for non-English languages in multilingual systems, offering a practical improvement for users of such translation tools.
The paper tackled the problem of low-quality non-English directions in multilingual machine translation by proposing mRASP2, a method using contrastive learning and data augmentation, which improved non-English translation by an average of over 10 BLEU points and achieved competitive performance on English-centric directions.
Existing multilingual machine translation approaches mainly focus on English-centric directions, while the non-English directions still lag behind. In this work, we aim to build a many-to-many translation system with an emphasis on the quality of non-English language directions. Our intuition is based on the hypothesis that a universal cross-language representation leads to better multilingual translation performance. To this end, we propose mRASP2, a training method to obtain a single unified multilingual translation model. mRASP2 is empowered by two techniques: a) a contrastive learning scheme to close the gap among representations of different languages, and b) data augmentation on both multiple parallel and monolingual data to further align token representations. For English-centric directions, mRASP2 outperforms existing best unified model and achieves competitive or even better performance than the pre-trained and fine-tuned model mBART on tens of WMT's translation directions. For non-English directions, mRASP2 achieves an improvement of average 10+ BLEU compared with the multilingual Transformer baseline. Code, data and trained models are available at https://github.com/PANXiao1994/mRASP2.