Beyond Disagreement-based Agnostic Active Learning
This addresses the challenge of efficient active learning without assumptions on label generation, which is incremental as it builds on prior work but offers broader applicability.
The paper tackles the problem of agnostic active learning by developing an algorithm that reduces label queries while being consistent and applicable to general classification, achieving improved label complexity compared to existing methods.
We study agnostic active learning, where the goal is to learn a classifier in a pre-specified hypothesis class interactively with as few label queries as possible, while making no assumptions on the true function generating the labels. The main algorithms for this problem are {\em{disagreement-based active learning}}, which has a high label requirement, and {\em{margin-based active learning}}, which only applies to fairly restricted settings. A major challenge is to find an algorithm which achieves better label complexity, is consistent in an agnostic setting, and applies to general classification problems. In this paper, we provide such an algorithm. Our solution is based on two novel contributions -- a reduction from consistent active learning to confidence-rated prediction with guaranteed error, and a novel confidence-rated predictor.