Pointer-based Fusion of Bilingual Lexicons into Neural Machine Translation
This work addresses challenges in NMT for low-resource language pairs, though it is incremental as it builds on existing fusion techniques.
The paper tackles the problem of improving neural machine translation (NMT) in low-resource scenarios by fusing bilingual lexicons with sequence-to-sequence models, resulting in models that reduce parameter size and training cost without sacrificing performance.
Neural machine translation (NMT) systems require large amounts of high quality in-domain parallel corpora for training. State-of-the-art NMT systems still face challenges related to out-of-vocabulary words and dealing with low-resource language pairs. In this paper, we propose and compare several models for fusion of bilingual lexicons with an end-to-end trained sequence-to-sequence model for machine translation. The result is a fusion model with two information sources for the decoder: a neural conditional language model and a bilingual lexicon. This fusion model learns how to combine both sources of information in order to produce higher quality translation output. Our experiments show that our proposed models work well in relatively low-resource scenarios, and also effectively reduce the parameter size and training cost for NMT without sacrificing performance.