Enhancing Cost Efficiency in Active Learning with Candidate Set Query
This work addresses the problem of cost efficiency in active learning for machine learning practitioners and researchers, offering an incremental improvement over traditional active learning methods.
This paper tackles the problem of reducing labeling cost in active learning, achieving a 48% reduction in labeling cost on ImageNet64x64. The framework introduces a novel query design that narrows down the set of candidate classes, significantly reducing the search space and labeling cost.
This paper introduces a cost-efficient active learning (AL) framework for classification, featuring a novel query design called candidate set query. Unlike traditional AL queries requiring the oracle to examine all possible classes, our method narrows down the set of candidate classes likely to include the ground-truth class, significantly reducing the search space and labeling cost. Moreover, we leverage conformal prediction to dynamically generate small yet reliable candidate sets, adapting to model enhancement over successive AL rounds. To this end, we introduce an acquisition function designed to prioritize data points that offer high information gain at lower cost. Empirical evaluations on CIFAR-10, CIFAR-100, and ImageNet64x64 demonstrate the effectiveness and scalability of our framework. Notably, it reduces labeling cost by 48% on ImageNet64x64. The project page can be found at https://yehogwon.github.io/csq-al.