RankAug: Augmented data ranking for text classification
This addresses the gap in synthetic data evaluation for NLU tasks like intent and sentiment classification, offering a method to enhance performance in under-represented classes.
The paper tackled the problem of evaluating synthetic data for text classification by proposing RankAug, a text-ranking approach that filters augmented texts based on similarity and diversity, resulting in up to 35% improvement in classification accuracy for under-represented classes.
Research on data generation and augmentation has been focused majorly on enhancing generation models, leaving a notable gap in the exploration and refinement of methods for evaluating synthetic data. There are several text similarity metrics within the context of generated data filtering which can impact the performance of specific Natural Language Understanding (NLU) tasks, specifically focusing on intent and sentiment classification. In this study, we propose RankAug, a text-ranking approach that detects and filters out the top augmented texts in terms of being most similar in meaning with lexical and syntactical diversity. Through experiments conducted on multiple datasets, we demonstrate that the judicious selection of filtering techniques can yield a substantial improvement of up to 35% in classification accuracy for under-represented classes.