Towards Principled Uncertainty Estimation for Deep Neural Networks
This work addresses the need for principled uncertainty estimation in deep learning, particularly for applications where misclassification costs are high, by providing a unified approach that handles multiple uncertainty types, though it is incremental as it builds on existing techniques.
The paper tackles the problem of accurate uncertainty estimation for deep neural networks in high-cost misclassification scenarios by introducing a unified hierarchical model that combines Bayesian inference, invertible latent density inference, and discriminative classification to address model capacity, intrinsic data, and open set uncertainties, achieving efficient per-sample uncertainty estimation for detection tasks.
When the cost of misclassifying a sample is high, it is useful to have an accurate estimate of uncertainty in the prediction for that sample. There are also multiple types of uncertainty which are best estimated in different ways, for example, uncertainty that is intrinsic to the training set may be well-handled by a Bayesian approach, while uncertainty introduced by shifts between training and query distributions may be better-addressed by density/support estimation. In this paper, we examine three types of uncertainty: model capacity uncertainty, intrinsic data uncertainty, and open set uncertainty, and review techniques that have been derived to address each one. We then introduce a unified hierarchical model, which combines methods from Bayesian inference, invertible latent density inference, and discriminative classification in a single end-to-end deep neural network topology to yield efficient per-sample uncertainty estimation in a detection context. This approach addresses all three uncertainty types and can readily accommodate prior/base rates for binary detection. We then discuss how to extend this model to a more generic multiclass recognition context.