Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding
This work addresses the inference efficiency problem for users of machine translation systems, though it is incremental as it adapts an existing technique to a specific domain.
The paper tackles the computational expense of Minimum Bayes Risk (MBR) decoding in neural machine translation by applying Direct Preference Optimization (DPO) to fine-tune Multilingual Large Language Models, achieving significantly improved performance on multiple test sets without extra inference cost.
Minimum Bayes Risk (MBR) decoding can significantly improve translation performance of Multilingual Large Language Models (MLLMs). However, MBR decoding is computationally expensive. We show how the recently developed Reinforcement Learning technique, Direct Preference Optimization (DPO), can fine-tune MLLMs to get the gains of MBR without any additional computation in inference. Our method uses only a small monolingual fine-tuning set and yields significantly improved performance on multiple NMT test sets compared to MLLMs without DPO.