Reinforcement Learning for Edit-Based Non-Autoregressive Neural Machine Translation
This work addresses efficiency and accuracy issues in machine translation for applications requiring low latency, but it is incremental as it builds on existing edit-based models.
The paper tackles the performance gap and training data challenges in non-autoregressive neural machine translation by applying reinforcement learning to an edit-based model, showing that RL with self-generated data improves performance, with specific gains such as a 1.5 BLEU score increase on WMT14 En-De.
Non-autoregressive (NAR) language models are known for their low latency in neural machine translation (NMT). However, a performance gap exists between NAR and autoregressive models due to the large decoding space and difficulty in capturing dependency between target words accurately. Compounding this, preparing appropriate training data for NAR models is a non-trivial task, often exacerbating exposure bias. To address these challenges, we apply reinforcement learning (RL) to Levenshtein Transformer, a representative edit-based NAR model, demonstrating that RL with self-generated data can enhance the performance of edit-based NAR models. We explore two RL approaches: stepwise reward maximization and episodic reward maximization. We discuss the respective pros and cons of these two approaches and empirically verify them. Moreover, we experimentally investigate the impact of temperature setting on performance, confirming the importance of proper temperature setting for NAR models' training.