Improving Retrieval Augmented Neural Machine Translation by Controlling Source and Fuzzy-Match Interactions
This work addresses domain adaptation challenges in machine translation for users needing efficient inference-time adaptation, though it is incremental as it builds on existing retrieval-augmented methods.
The paper tackled the problem of zero-shot adaptation in neural machine translation by proposing a novel architecture to control interactions between source sentences and retrieved fuzzy-match translations, resulting in consistent BLEU score improvements across multiple language pairs and domains.
We explore zero-shot adaptation, where a general-domain model has access to customer or domain specific parallel data at inference time, but not during training. We build on the idea of Retrieval Augmented Translation (RAT) where top-k in-domain fuzzy matches are found for the source sentence, and target-language translations of those fuzzy-matched sentences are provided to the translation model at inference time. We propose a novel architecture to control interactions between a source sentence and the top-k fuzzy target-language matches, and compare it to architectures from prior work. We conduct experiments in two language pairs (En-De and En-Fr) by training models on WMT data and testing them with five and seven multi-domain datasets, respectively. Our approach consistently outperforms the alternative architectures, improving BLEU across language pair, domain, and number k of fuzzy matches.