Scalable Batch Acquisition for Deep Bayesian Active Learning
This addresses a scalability bottleneck for researchers and practitioners using Bayesian active learning on large datasets, though it is an incremental improvement over existing methods.
The paper tackles the computational inefficiency of existing Bayesian active learning methods when selecting large batches of examples, presenting Large BatchBALD as an approximation that reduces computation time while maintaining comparable quality, with experiments showing significant speed improvements on datasets like CIFAR-100.
In deep active learning, it is especially important to choose multiple examples to markup at each step to work efficiently, especially on large datasets. At the same time, existing solutions to this problem in the Bayesian setup, such as BatchBALD, have significant limitations in selecting a large number of examples, associated with the exponential complexity of computing mutual information for joint random variables. We, therefore, present the Large BatchBALD algorithm, which gives a well-grounded approximation to the BatchBALD method that aims to achieve comparable quality while being more computationally efficient. We provide a complexity analysis of the algorithm, showing a reduction in computation time, especially for large batches. Furthermore, we present an extensive set of experimental results on image and text data, both on toy datasets and larger ones such as CIFAR-100.