A Case Study on Context Encoding in Multi-Encoder based Document-Level Neural Machine Translation
This work addresses pronoun translation accuracy in machine translation, but it is incremental as it builds on existing multi-encoder models.
The study investigated how different context settings affect pronoun translation accuracy in multi-encoder document-level neural machine translation, finding that models perform well even with random context and that mixing selected and random contexts yields the best results.
Recent studies have shown that the multi-encoder models are agnostic to the choice of context, and the context encoder generates noise which helps improve the models in terms of BLEU score. In this paper, we further explore this idea by evaluating with context-aware pronoun translation test set by training multi-encoder models trained on three different context settings viz, previous two sentences, random two sentences, and a mix of both as context. Specifically, we evaluate the models on the ContraPro test set to study how different contexts affect pronoun translation accuracy. The results show that the model can perform well on the ContraPro test set even when the context is random. We also analyze the source representations to study whether the context encoder generates noise. Our analysis shows that the context encoder provides sufficient information to learn discourse-level information. Additionally, we observe that mixing the selected context (the previous two sentences in this case) and the random context is generally better than the other settings.