Convergence of Uncertainty Sampling for Active Learning
This work addresses the problem of reducing annotation costs in active learning for practitioners, offering theoretical foundations for uncertainty sampling methods.
The paper tackles the lack of convergence guarantees for uncertainty sampling in active learning by proposing an efficient uncertainty estimator for binary and multi-class classification, providing non-asymptotic convergence rates under no-noise conditions and theoretical guarantees for noisy cases.
Uncertainty sampling in active learning is heavily used in practice to reduce the annotation cost. However, there has been no wide consensus on the function to be used for uncertainty estimation in binary classification tasks and convergence guarantees of the corresponding active learning algorithms are not well understood. The situation is even more challenging for multi-category classification. In this work, we propose an efficient uncertainty estimator for binary classification which we also extend to multiple classes, and provide a non-asymptotic rate of convergence for our uncertainty sampling-based active learning algorithm in both cases under no-noise conditions (i.e., linearly separable data). We also extend our analysis to the noisy case and provide theoretical guarantees for our algorithm under the influence of noise in the task of binary and multi-class classification.