Document-Level Neural Machine Translation with Hierarchical Attention Networks
This work addresses the need for better translation quality in NLP by leveraging document-level information, though it is incremental as it builds on existing NMT architectures.
The paper tackled the problem of improving neural machine translation by incorporating document-level context, resulting in a significant BLEU score improvement over a strong baseline and state-of-the-art context-aware methods.
Neural Machine Translation (NMT) can be improved by including document-level contextual information. For this purpose, we propose a hierarchical attention model to capture the context in a structured and dynamic manner. The model is integrated in the original NMT architecture as another level of abstraction, conditioning on the NMT model's own previous hidden states. Experiments show that hierarchical attention significantly improves the BLEU score over a strong NMT baseline with the state-of-the-art in context-aware methods, and that both the encoder and decoder benefit from context in complementary ways.