Importance-Aware Data Augmentation for Document-Level Neural Machine Translation
This addresses data scarcity for document-level translation, but it is incremental as it builds on existing DocNMT methods with a new augmentation technique.
The paper tackles data sparsity in document-level neural machine translation (DocNMT) by proposing an Importance-Aware Data Augmentation (IADA) algorithm that uses token importance from hidden states and gradients, and it shows statistically significant improvements in BLEU scores over baselines on three benchmarks.
Document-level neural machine translation (DocNMT) aims to generate translations that are both coherent and cohesive, in contrast to its sentence-level counterpart. However, due to its longer input length and limited availability of training data, DocNMT often faces the challenge of data sparsity. To overcome this issue, we propose a novel Importance-Aware Data Augmentation (IADA) algorithm for DocNMT that augments the training data based on token importance information estimated by the norm of hidden states and training gradients. We conduct comprehensive experiments on three widely-used DocNMT benchmarks. Our empirical results show that our proposed IADA outperforms strong DocNMT baselines as well as several data augmentation approaches, with statistical significance on both sentence-level and document-level BLEU.