AdMix: A Mixed Sample Data Augmentation Method for Neural Machine Translation
This work addresses data augmentation for neural machine translation, offering an incremental improvement over existing methods like back-translation.
The paper tackles the problem of improving neural machine translation performance by proposing AdMix, a data augmentation method that introduces noise and mixes samples without extra data, achieving improvements of 1.0 to 2.7 BLEU points over a Transformer baseline.
In Neural Machine Translation (NMT), data augmentation methods such as back-translation have proven their effectiveness in improving translation performance. In this paper, we propose a novel data augmentation approach for NMT, which is independent of any additional training data. Our approach, AdMix, consists of two parts: 1) introduce faint discrete noise (word replacement, word dropping, word swapping) into the original sentence pairs to form augmented samples; 2) generate new synthetic training data by softly mixing the augmented samples with their original samples in training corpus. Experiments on three translation datasets of different scales show that AdMix achieves signifi cant improvements (1.0 to 2.7 BLEU points) over strong Transformer baseline. When combined with other data augmentation techniques (e.g., back-translation), our approach can obtain further improvements.