On the QUEST for Uncertainty Quantification via Highest Density Regions
For probabilistic machine learning practitioners, QUEST provides a more principled and robust way to quantify epistemic and aleatoric uncertainty that avoids counterintuitive results of standard approaches.
The paper proposes QUEST, a new framework for uncertainty quantification in regression that uses the volume of highest density regions instead of pointwise predictive risk. The method satisfies desired axioms and outperforms variance and differential entropy on selective prediction benchmarks.
Uncertainty quantification (UQ) is essential for reliable decision-making in safety-critical applications in probabilistic machine learning. For regression problems, dominant scalar UQ approaches - notably, those based on proper scoring rules - measure uncertainty via pointwise predictive risk. This can lead to counterintuitive results when the target statistic is not the conditional expectation. We propose an alternative framework, in which uncertainty is characterised by the volume of the most probable subset of a distribution's support. QUEST (Quantifying Uncertainty via highest dEnSiTy regions) is a novel approach to UQ based on the concentration of Lebesgue measure at a distribution's peak(s), evaluated at one or more values of a robustness parameter $α$. We establish connections between our measures and classical statistics from information theory and economics. We show that, unlike popular alternatives based on proper scoring rules, QUEST measures of epistemic and aleatoric uncertainty satisfy a set of axioms adapted from the UQ literature, including monotonicity under distributional spread and invariance to location shifts. Selective prediction benchmarks confirm that QUEST performs favourably against standard measures such as variance and differential entropy.