Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoders
This addresses the challenge of flexible and efficient extension to new languages in multilingual machine translation, representing an incremental improvement over existing methods.
The paper tackles the problem of retraining entire systems to add new languages in multilingual machine translation by proposing language-specific encoder-decoders, which outperform a universal encoder-decoder by 3.28 BLEU points on average and allow extension without retraining other modules.
State-of-the-art multilingual machine translation relies on a universal encoder-decoder, which requires retraining the entire system to add new languages. In this paper, we propose an alternative approach that is based on language-specific encoder-decoders, and can thus be more easily extended to new languages by learning their corresponding modules. So as to encourage a common interlingua representation, we simultaneously train the N initial languages. Our experiments show that the proposed approach outperforms the universal encoder-decoder by 3.28 BLEU points on average, and when adding new languages, without the need to retrain the rest of the modules. All in all, our work closes the gap between shared and language-specific encoder-decoders, advancing toward modular multilingual machine translation systems that can be flexibly extended in lifelong learning settings.