Improving Robustness of Retrieval Augmented Translation via Shuffling of Suggestions
This addresses robustness issues in neural machine translation for users relying on translation memories, though it is incremental as it builds on existing fuzzy-match methods.
The paper tackles the problem of retrieval augmented translation systems performing poorly when translation memories have domain mismatches, and shows that a simple training method can regain up to 5.8 BLEU points while maintaining competitiveness with relevant data.
Several recent studies have reported dramatic performance improvements in neural machine translation (NMT) by augmenting translation at inference time with fuzzy-matches retrieved from a translation memory (TM). However, these studies all operate under the assumption that the TMs available at test time are highly relevant to the testset. We demonstrate that for existing retrieval augmented translation methods, using a TM with a domain mismatch to the test set can result in substantially worse performance compared to not using a TM at all. We propose a simple method to expose fuzzy-match NMT systems during training and show that it results in a system that is much more tolerant (regaining up to 5.8 BLEU) to inference with TMs with domain mismatch. Also, the model is still competitive to the baseline when fed with suggestions from relevant TMs.