BET: A Backtranslation Approach for Easy Data Augmentation in Transformer-based Paraphrase Identification Context
This work addresses the problem of data scarcity and generalization for researchers and practitioners using transformer models in NLP, offering an incremental improvement through a cost-effective augmentation technique.
The paper tackles the fixed size and limited generalizability of datasets for transformer-based NLP models by proposing BET, a backtranslation data augmentation approach, which improves paraphrase identification performance on the MRPC dataset by over 3% in accuracy and F1 score and enables reasonable performance in low-data regimes.
Newly-introduced deep learning architectures, namely BERT, XLNet, RoBERTa and ALBERT, have been proved to be robust on several NLP tasks. However, the datasets trained on these architectures are fixed in terms of size and generalizability. To relieve this issue, we apply one of the most inexpensive solutions to update these datasets. We call this approach BET by which we analyze the backtranslation data augmentation on the transformer-based architectures. Using the Google Translate API with ten intermediary languages from ten different language families, we externally evaluate the results in the context of automatic paraphrase identification in a transformer-based framework. Our findings suggest that BET improves the paraphrase identification performance on the Microsoft Research Paraphrase Corpus (MRPC) to more than 3% on both accuracy and F1 score. We also analyze the augmentation in the low-data regime with downsampled versions of MRPC, Twitter Paraphrase Corpus (TPC) and Quora Question Pairs. In many low-data cases, we observe a switch from a failing model on the test set to reasonable performances. The results demonstrate that BET is a highly promising data augmentation technique: to push the current state-of-the-art of existing datasets and to bootstrap the utilization of deep learning architectures in the low-data regime of a hundred samples.