Agreement-based Joint Training for Bidirectional Attention-based Neural Machine Translation
This work addresses translation quality issues in neural machine translation for language pairs like Chinese-English and English-French, but it is incremental as it builds on existing bidirectional attention methods.
The paper tackled the problem of unidirectional attention models capturing only partial attentional regularities in neural machine translation by proposing agreement-based joint training for bidirectional attention models, which improved alignment and translation quality on Chinese-English and English-French tasks.
The attentional mechanism has proven to be effective in improving end-to-end neural machine translation. However, due to the intricate structural divergence between natural languages, unidirectional attention-based models might only capture partial aspects of attentional regularities. We propose agreement-based joint training for bidirectional attention-based end-to-end neural machine translation. Instead of training source-to-target and target-to-source translation models independently,our approach encourages the two complementary models to agree on word alignment matrices on the same training data. Experiments on Chinese-English and English-French translation tasks show that agreement-based joint training significantly improves both alignment and translation quality over independent training.