Decoding-History-Based Adaptive Control of Attention for Neural Machine Translation
This work addresses repetition issues in neural machine translation, which is an incremental improvement for translation systems.
The paper tackled the problem of repetition in neural machine translation by proposing a decoding-history-based adaptive control of attention, which significantly outperformed strong baselines on Chinese-English and English-Vietnamese translation tasks.
Attention-based sequence-to-sequence model has proved successful in Neural Machine Translation (NMT). However, the attention without consideration of decoding history, which includes the past information in the decoder and the attention mechanism, often causes much repetition. To address this problem, we propose the decoding-history-based Adaptive Control of Attention (ACA) for the NMT model. ACA learns to control the attention by keeping track of the decoding history and the current information with a memory vector, so that the model can take the translated contents and the current information into consideration. Experiments on Chinese-English translation and the English-Vietnamese translation have demonstrated that our model significantly outperforms the strong baselines. The analysis shows that our model is capable of generating translation with less repetition and higher accuracy. The code will be available at https://github.com/lancopku