Reducing and Exploiting Data Augmentation Noise through Meta Reweighting Contrastive Learning for Text Classification
This work addresses the data-hungry problem in text classification by improving model performance with augmented data, though it is incremental as it builds on existing augmentation and learning techniques.
The paper tackles the problem of varying quality in augmented text data by proposing a meta reweighting contrastive learning framework that assigns weights to augmented samples based on their quality, achieving average absolute improvements of 1.6% on Text-CNN and 1.4% on RoBERTa-base across seven GLUE datasets.
Data augmentation has shown its effectiveness in resolving the data-hungry problem and improving model's generalization ability. However, the quality of augmented data can be varied, especially compared with the raw/original data. To boost deep learning models' performance given augmented data/samples in text classification tasks, we propose a novel framework, which leverages both meta learning and contrastive learning techniques as parts of our design for reweighting the augmented samples and refining their feature representations based on their quality. As part of the framework, we propose novel weight-dependent enqueue and dequeue algorithms to utilize augmented samples' weight/quality information effectively. Through experiments, we show that our framework can reasonably cooperate with existing deep learning models (e.g., RoBERTa-base and Text-CNN) and augmentation techniques (e.g., Wordnet and Easydata) for specific supervised learning tasks. Experiment results show that our framework achieves an average of 1.6%, up to 4.3% absolute improvement on Text-CNN encoders and an average of 1.4%, up to 4.4% absolute improvement on RoBERTa-base encoders on seven GLUE benchmark datasets compared with the best baseline. We present an indepth analysis of our framework design, revealing the non-trivial contributions of our network components. Our code is publicly available for better reproducibility.