Black-Box Batch Active Learning for Regression
This work addresses the limitation of existing batch active learning methods that are often restricted to differentiable models, benefiting researchers and practitioners using diverse machine learning models.
The paper tackles the problem of batch active learning for regression by proposing a black-box method that relies only on model predictions, making it compatible with a wide range of models including non-differentiable ones, and it achieves strong performance comparable to white-box approaches in experiments.
Batch active learning is a popular approach for efficiently training machine learning models on large, initially unlabelled datasets by repeatedly acquiring labels for batches of data points. However, many recent batch active learning methods are white-box approaches and are often limited to differentiable parametric models: they score unlabeled points using acquisition functions based on model embeddings or first- and second-order derivatives. In this paper, we propose black-box batch active learning for regression tasks as an extension of white-box approaches. Crucially, our method only relies on model predictions. This approach is compatible with a wide range of machine learning models, including regular and Bayesian deep learning models and non-differentiable models such as random forests. It is rooted in Bayesian principles and utilizes recent kernel-based approaches. This allows us to extend a wide range of existing state-of-the-art white-box batch active learning methods (BADGE, BAIT, LCMD) to black-box models. We demonstrate the effectiveness of our approach through extensive experimental evaluations on regression datasets, achieving surprisingly strong performance compared to white-box approaches for deep learning models.