Leveraging Variation Theory in Counterfactual Data Augmentation for Optimized Active Learning
This work addresses the cold start problem in active learning for text classification, though it appears incremental as it builds on existing methods with a novel hybrid approach.
The paper tackled the problem of selecting datapoints for user querying in active learning by introducing a counterfactual data augmentation approach inspired by Variation Theory, achieving significantly higher performance with fewer annotated data in text classification experiments.
Active Learning (AL) allows models to learn interactively from user feedback. This paper introduces a counterfactual data augmentation approach to AL, particularly addressing the selection of datapoints for user querying, a pivotal concern in enhancing data efficiency. Our approach is inspired by Variation Theory, a theory of human concept learning that emphasizes the essential features of a concept by focusing on what stays the same and what changes. Instead of just querying with existing datapoints, our approach synthesizes artificial datapoints that highlight potential key similarities and differences among labels using a neuro-symbolic pipeline combining large language models (LLMs) and rule-based models. Through an experiment in the example domain of text classification, we show that our approach achieves significantly higher performance when there are fewer annotated data. As the annotated training data gets larger the impact of the generated data starts to diminish showing its capability to address the cold start problem in AL. This research sheds light on integrating theories of human learning into the optimization of AL.