Learning Policies for Multilingual Training of Neural Machine Translation Systems
This work addresses the challenge of enhancing translation quality for low-resource languages by leveraging high-resource data, but it is incremental as it builds on existing techniques like fine-tuning.
The paper tackled the problem of improving translation performance for low-resource languages in multilingual neural machine translation by proposing search-based curricula and learning curricula from scratch using contextual multi-arm bandits, showing that these methods provide better starting points for fine-tuning and improve overall system performance on the FLORES dataset.
Low-resource Multilingual Neural Machine Translation (MNMT) is typically tasked with improving the translation performance on one or more language pairs with the aid of high-resource language pairs. In this paper, we propose two simple search based curricula -- orderings of the multilingual training data -- which help improve translation performance in conjunction with existing techniques such as fine-tuning. Additionally, we attempt to learn a curriculum for MNMT from scratch jointly with the training of the translation system with the aid of contextual multi-arm bandits. We show on the FLORES low-resource translation dataset that these learned curricula can provide better starting points for fine tuning and improve overall performance of the translation system.