Do you understand epistemic uncertainty? Think again! Rigorous frequentist epistemic uncertainty estimation in regression
This work addresses uncertainty quantification for regression models, which is critical for reliable predictions in applications like autonomous systems or healthcare, though it builds incrementally on prior classification methods.
The paper tackles the challenge of distinguishing between aleatoric and epistemic uncertainty in regression by extending a frequentist approach from classification, resulting in a method that provides rigorous uncertainty estimation with minimal architectural changes.
Quantifying model uncertainty is critical for understanding prediction reliability, yet distinguishing between aleatoric and epistemic uncertainty remains challenging. We extend recent work from classification to regression to provide a novel frequentist approach to epistemic and aleatoric uncertainty estimation. We train models to generate conditional predictions by feeding their initial output back as an additional input. This method allows for a rigorous measurement of model uncertainty by observing how prediction responses change when conditioned on the model's previous answer. We provide a complete theoretical framework to analyze epistemic uncertainty in regression in a frequentist way, and explain how it can be exploited in practice to gauge a model's uncertainty, with minimal changes to the original architecture.