Exponential Savings in Agnostic Active Learning through Abstention
This work addresses label efficiency in active learning for practitioners, but it is incremental as it extends known results to a more general setting with abstention.
The paper tackles the problem of reducing label requests in pool-based active classification without distributional assumptions by allowing the learner to abstain from predictions at a cost slightly below random guessing, achieving exponential savings in label requests when possible in the realizable case.
We show that in pool-based active classification without assumptions on the underlying distribution, if the learner is given the power to abstain from some predictions by paying the price marginally smaller than the average loss $1/2$ of a random guess, exponential savings in the number of label requests are possible whenever they are possible in the corresponding realizable problem. We extend this result to provide a necessary and sufficient condition for exponential savings in pool-based active classification under the model misspecification.