Domain Adaptive Inference for Neural Machine Translation
This work addresses domain adaptation in machine translation, which is incremental as it builds on existing ensemble and regularization methods.
The paper tackles the problem of improving Neural Machine Translation performance on new domains without degrading original domain performance, by introducing an adaptive ensemble decoding scheme that uses Bayesian Interpolation with source information, achieving strong improvements across test domains without domain labels.
We investigate adaptive ensemble weighting for Neural Machine Translation, addressing the case of improving performance on a new and potentially unknown domain without sacrificing performance on the original domain. We adapt sequentially across two Spanish-English and three English-German tasks, comparing unregularized fine-tuning, L2 and Elastic Weight Consolidation. We then report a novel scheme for adaptive NMT ensemble decoding by extending Bayesian Interpolation with source information, and show strong improvements across test domains without access to the domain label.