MALM: Mixing Augmented Language Modeling for Zero-Shot Machine Translation
This addresses a key challenge in zero-shot multilingual machine translation for NLP applications, though it appears incremental as it builds on existing pre-training and augmentation methods.
The paper tackles the problem of off-target language errors in zero-shot machine translation by using prompt-conditioned large models, demonstrating that they avoid translating to wrong languages and achieve effective results through self-supervised pre-training and data augmentation.
Large pre-trained language models have brought remarkable progress in NLP. Pre-training and Fine-tuning have given state-of-art performance across tasks in text processing. Data Augmentation techniques have also helped build state-of-art models on low or zero resource tasks. Many works in the past have attempted at learning a single massively-multilingual machine translation model for zero-shot translation. Although those translation models are producing correct translations, the main challenge is those models are producing the wrong languages for zero-shot translation. This work and its results indicate that prompt conditioned large models do not suffer from off-target language errors i.e. errors arising due to translation to wrong languages. We empirically demonstrate the effectiveness of self-supervised pre-training and data augmentation for zero-shot multi-lingual machine translation.