Multiple References with Meaningful Variations Improve Literary Machine Translation
This work addresses the challenge of limited reference diversity in machine translation for literary texts, offering incremental improvements in translation quality.
The paper tackled the problem of improving literary machine translation by using multiple reference translations with varying semantic similarity, finding that fine-tuning large language models with medium and high similarity paraphrases led to gains in BLEU (0.3-0.5), COMET (0.1-0.9), and chrF++ (0.17-0.32) scores.
While a source sentence can be translated in many ways, most machine translation (MT) models are trained with only a single reference. Previous work has shown that using synthetic paraphrases can improve MT. This paper investigates best practices for employing multiple references by analyzing the semantic similarity among different English translations of world literature in the Par3 dataset. We classify the semantic similarity between paraphrases into three levels: low, medium, and high, and fine-tune three different models (mT5-large, LLaMA-2-7B, and Opus-MT) for literary MT tasks. Across different models, holding the total training instances constant, single-reference but more source texts only marginally outperforms multiple-reference with half of the source texts. Moreover, when fine-tuning an LLM, using paraphrases with medium and high semantic similarity outperforms an unfiltered dataset, with improvements in BLEU (0.3-0.5), COMET (0.1-0.9), and chrF++ (0.17-0.32). Our code is publicly available on GitHub.