Domain Adaptation of Neural Machine Translation by Lexicon Induction
This addresses domain adaptation for neural machine translation, which is an incremental improvement over existing methods.
The paper tackled the problem of neural machine translation being sensitive to domain shift by proposing an unsupervised adaptation method using lexicon induction and pseudo-parallel corpus construction, achieving improvements of up to 14 BLEU over unadapted models and up to 2 BLEU over baselines across multiple domains and settings.
It has been previously noted that neural machine translation (NMT) is very sensitive to domain shift. In this paper, we argue that this is a dual effect of the highly lexicalized nature of NMT, resulting in failure for sentences with large numbers of unknown words, and lack of supervision for domain-specific words. To remedy this problem, we propose an unsupervised adaptation method which fine-tunes a pre-trained out-of-domain NMT model using a pseudo-in-domain corpus. Specifically, we perform lexicon induction to extract an in-domain lexicon, and construct a pseudo-parallel in-domain corpus by performing word-for-word back-translation of monolingual in-domain target sentences. In five domains over twenty pairwise adaptation settings and two model architectures, our method achieves consistent improvements without using any in-domain parallel sentences, improving up to 14 BLEU over unadapted models, and up to 2 BLEU over strong back-translation baselines.