Towards one-shot learning for rare-word translation with external experts
This addresses translation quality issues for rare words in NMT systems, particularly in out-of-domain settings, but is incremental as it builds on existing methods like pointer networks and reinforcement learning.
The paper tackles the problem of translating rare words in neural machine translation by using external experts to annotate training data and controlling interactions with a pointer network and reinforcement learning, resulting in improvements of over 1.0 BLEU point in out-of-domain scenarios for English to Spanish and German to English.
Neural machine translation (NMT) has significantly improved the quality of automatic translation models. One of the main challenges in current systems is the translation of rare words. We present a generic approach to address this weakness by having external models annotate the training data as Experts, and control the model-expert interaction with a pointer network and reinforcement learning. Our experiments using phrase-based models to simulate Experts to complement neural machine translation models show that the model can be trained to copy the annotations into the output consistently. We demonstrate the benefit of our proposed framework in outof-domain translation scenarios with only lexical resources, improving more than 1.0 BLEU point in both translation directions English to Spanish and German to English