Deep Probabilistic Ensembles: Approximate Variational Inference through KL Regularization
This work addresses uncertainty estimation for active learning in visual classification, presenting an incremental improvement over existing methods.
The paper tackles the problem of approximating deep Bayesian Neural Networks with a scalable technique called Deep Probabilistic Ensembles, using KL regularization derived from variational inference, and shows that it steadily improves active learning baselines on visual classification as annotation budget increases.
In this paper, we introduce Deep Probabilistic Ensembles (DPEs), a scalable technique that uses a regularized ensemble to approximate a deep Bayesian Neural Network (BNN). We do so by incorporating a KL divergence penalty term into the training objective of an ensemble, derived from the evidence lower bound used in variational inference. We evaluate the uncertainty estimates obtained from our models for active learning on visual classification. Our approach steadily improves upon active learning baselines as the annotation budget is increased.