Uncertainty Estimation using Variance-Gated Distributions
This work addresses uncertainty estimation for high-risk applications, but it appears incremental as it builds on existing decomposition methods.
The paper tackles the problem of per-sample uncertainty quantification in neural networks by proposing a variance-gated framework based on signal-to-noise ratios, and it discusses the collapse in diversity of committee machines as a result.
Evaluation of per-sample uncertainty quantification from neural networks is essential for decision-making involving high-risk applications. A common approach is to use the predictive distribution from Bayesian or approximation models and decompose the corresponding predictive uncertainty into epistemic (model-related) and aleatoric (data-related) components. However, additive decomposition has recently been questioned. In this work, we propose an intuitive framework for uncertainty estimation and decomposition based on the signal-to-noise ratio of class probability distributions across different model predictions. We introduce a variance-gated measure that scales predictions by a confidence factor derived from ensembles. We use this measure to discuss the existence of a collapse in the diversity of committee machines.