When and Why is Document-level Context Useful in Neural Machine Translation?
This work clarifies the limits of document-level context for NMT researchers, showing it is often incremental and not broadly effective.
The study investigated the utility of document-level context in neural machine translation, finding that most improvements are not due to context utilization and that minimal encoding suffices, with long context being unhelpful.
Document-level context has received lots of attention for compensating neural machine translation (NMT) of isolated sentences. However, recent advances in document-level NMT focus on sophisticated integration of the context, explaining its improvement with only a few selected examples or targeted test sets. We extensively quantify the causes of improvements by a document-level model in general test sets, clarifying the limit of the usefulness of document-level context in NMT. We show that most of the improvements are not interpretable as utilizing the context. We also show that a minimal encoding is sufficient for the context modeling and very long context is not helpful for NMT.