CipherDAug: Ciphertext based Data Augmentation for Neural Machine Translation
This addresses data scarcity for neural machine translation practitioners, offering an incremental improvement over current augmentation techniques.
The paper tackles the problem of data scarcity in neural machine translation by proposing CipherDAug, a data-augmentation technique using ROT-k ciphertexts on source text, which significantly outperforms existing methods on multiple datasets, especially in low-resource settings.
We propose a novel data-augmentation technique for neural machine translation based on ROT-$k$ ciphertexts. ROT-$k$ is a simple letter substitution cipher that replaces a letter in the plaintext with the $k$th letter after it in the alphabet. We first generate multiple ROT-$k$ ciphertexts using different values of $k$ for the plaintext which is the source side of the parallel data. We then leverage this enciphered training data along with the original parallel data via multi-source training to improve neural machine translation. Our method, CipherDAug, uses a co-regularization-inspired training procedure, requires no external data sources other than the original training data, and uses a standard Transformer to outperform strong data augmentation techniques on several datasets by a significant margin. This technique combines easily with existing approaches to data augmentation, and yields particularly strong results in low-resource settings.