Batch Active Learning from the Perspective of Sparse Approximation
This work addresses efficient model training for machine learning practitioners by providing an incremental improvement in batch active learning methods.
The paper tackles the problem of batch active learning by formulating it as a sparse approximation task, aiming to select an informative subset of unlabeled data to approximate the full data training loss, and achieves competitive performance with lower computational complexity in experiments.
Active learning enables efficient model training by leveraging interactions between machine learning agents and human annotators. We study and propose a novel framework that formulates batch active learning from the sparse approximation's perspective. Our active learning method aims to find an informative subset from the unlabeled data pool such that the corresponding training loss function approximates its full data pool counterpart. We realize the framework as sparsity-constrained discontinuous optimization problems, which explicitly balance uncertainty and representation for large-scale applications and could be solved by greedy or proximal iterative hard thresholding algorithms. The proposed method can adapt to various settings, including both Bayesian and non-Bayesian neural networks. Numerical experiments show that our work achieves competitive performance across different settings with lower computational complexity.