Supervised Attentions for Neural Machine Translation
This addresses the challenge of alignment accuracy in neural machine translation, offering a method to enhance translation quality, though it is incremental as it builds on existing attention mechanisms.
The paper tackled the problem of improving attention accuracy in neural machine translation by incorporating true alignments from training data, resulting in significant improvements in both translation and alignment quality, including beating a state-of-the-art syntax-based system on a Chinese-to-English task.
In this paper, we improve the attention or alignment accuracy of neural machine translation by utilizing the alignments of training sentence pairs. We simply compute the distance between the machine attentions and the "true" alignments, and minimize this cost in the training procedure. Our experiments on large-scale Chinese-to-English task show that our model improves both translation and alignment qualities significantly over the large-vocabulary neural machine translation system, and even beats a state-of-the-art traditional syntax-based system.