From Risk to Uncertainty: Generating Predictive Uncertainty Measures via Bayesian Estimation
This work addresses the need for clearer predictive uncertainty measures in machine learning, particularly for tasks like anomaly detection, but it is incremental as it builds on existing decomposition and Bayesian methods.
The paper tackles the problem of unclear relationships between predictive uncertainty measures by decomposing statistical pointwise risk into aleatoric and epistemic uncertainty components, using Bayesian methods to generate these measures, and validates them on image datasets with AUROC metrics for detecting out-of-distribution and misclassified instances, showing they are useful for these tasks.
There are various measures of predictive uncertainty in the literature, but their relationships to each other remain unclear. This paper uses a decomposition of statistical pointwise risk into components, associated with different sources of predictive uncertainty, namely aleatoric uncertainty (inherent data variability) and epistemic uncertainty (model-related uncertainty). Together with Bayesian methods, applied as an approximation, we build a framework that allows one to generate different predictive uncertainty measures. We validate our method on image datasets by evaluating its performance in detecting out-of-distribution and misclassified instances using the AUROC metric. The experimental results confirm that the measures derived from our framework are useful for the considered downstream tasks.