Distributionally Robust Multilingual Machine Translation
This work addresses the challenge of uniform performance across languages in multilingual translation models, which is crucial for deploying efficient and accurate translation systems, though it is incremental as it builds on existing optimization techniques.
The paper tackles the problem of data imbalance in multilingual neural machine translation by proposing a distributionally robust optimization objective that minimizes the worst-case expected loss across language pairs, resulting in consistent performance improvements in average and per-language translation accuracy across multiple datasets and settings.
Multilingual neural machine translation (MNMT) learns to translate multiple language pairs with a single model, potentially improving both the accuracy and the memory-efficiency of deployed models. However, the heavy data imbalance between languages hinders the model from performing uniformly across language pairs. In this paper, we propose a new learning objective for MNMT based on distributionally robust optimization, which minimizes the worst-case expected loss over the set of language pairs. We further show how to practically optimize this objective for large translation corpora using an iterated best response scheme, which is both effective and incurs negligible additional computational cost compared to standard empirical risk minimization. We perform extensive experiments on three sets of languages from two datasets and show that our method consistently outperforms strong baseline methods in terms of average and per-language performance under both many-to-one and one-to-many translation settings.