Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents
This addresses the data scarcity problem for document-level translation in low-resource languages, though it is incremental as it builds on existing multilingual and transfer learning methods.
The paper tackles the shortage of parallel documents for document-level neural machine translation by studying zero-shot transfer from languages with document data to those with only sentence data, showing feasibility with multilingual modeling and observing that more teacher languages and balanced data improve transfer quality, achieving decent performance on document-specific metrics.
Document-level neural machine translation (DocNMT) achieves coherent translations by incorporating cross-sentence context. However, for most language pairs there's a shortage of parallel documents, although parallel sentences are readily available. In this paper, we study whether and how contextual modeling in DocNMT is transferable via multilingual modeling. We focus on the scenario of zero-shot transfer from teacher languages with document level data to student languages with no documents but sentence level data, and for the first time treat document-level translation as a transfer learning problem. Using simple concatenation-based DocNMT, we explore the effect of 3 factors on the transfer: the number of teacher languages with document level data, the balance between document and sentence level data at training, and the data condition of parallel documents (genuine vs. backtranslated). Our experiments on Europarl-7 and IWSLT-10 show the feasibility of multilingual transfer for DocNMT, particularly on document-specific metrics. We observe that more teacher languages and adequate data balance both contribute to better transfer quality. Surprisingly, the transfer is less sensitive to the data condition, where multilingual DocNMT delivers decent performance with either backtranslated or genuine document pairs.