Uncertainty Quantification for Regression using Proper Scoring Rules
This work addresses the problem of reliable uncertainty estimation in regression for safety-critical applications, representing an incremental extension of existing UQ theory from classification.
The authors tackled the challenge of extending uncertainty quantification (UQ) from classification to regression by introducing a unified framework based on proper scoring rules, resulting in derived closed-form uncertainty measures that decompose into aleatoric and epistemic components and recover existing methods like predictive variance and differential entropy.
Quantifying uncertainty of machine learning model predictions is essential for reliable decision-making, especially in safety-critical applications. Recently, uncertainty quantification (UQ) theory has advanced significantly, building on a firm basis of learning with proper scoring rules. However, these advances were focused on classification, while extending these ideas to regression remains challenging. In this work, we introduce a unified UQ framework for regression based on proper scoring rules, such as CRPS, logarithmic, squared error, and quadratic scores. We derive closed-form expressions for the resulting uncertainty measures under practical parametric assumptions and show how to estimate them using ensembles of models. In particular, the derived uncertainty measures naturally decompose into aleatoric and epistemic components. The framework recovers popular regression UQ measures based on predictive variance and differential entropy. Our broad evaluation on synthetic and real-world regression datasets provides guidance for selecting reliable UQ measures.