Speeding Up BatchBALD: A k-BALD Family of Approximations for Active Learning
This work addresses efficiency issues in active learning for practitioners dealing with large datasets, though it is incremental as it builds on existing BatchBALD methods.
The paper tackles the computational slowness of BatchBALD in active learning by proposing k-BALD, a faster approximation using k-wise mutual information, which on MNIST achieves similar performance with significant speed improvements.
Active learning is a powerful method for training machine learning models with limited labeled data. One commonly used technique for active learning is BatchBALD, which uses Bayesian neural networks to find the most informative points to label in a pool set. However, BatchBALD can be very slow to compute, especially for larger datasets. In this paper, we propose a new approximation, k-BALD, which uses k-wise mutual information terms to approximate BatchBALD, making it much less expensive to compute. Results on the MNIST dataset show that k-BALD is significantly faster than BatchBALD while maintaining similar performance. Additionally, we also propose a dynamic approach for choosing k based on the quality of the approximation, making it more efficient for larger datasets.