Second-Order Uncertainty Quantification: Variance-Based Measures
This work addresses uncertainty quantification for machine learning practitioners in classification tasks, offering a novel approach that is incremental in building on existing methods.
The paper tackles uncertainty quantification in classification by proposing variance-based measures derived from second-order distributions, which enable class-level uncertainty reasoning and are shown empirically to be effective and competitive with entropy-based measures.
Uncertainty quantification is a critical aspect of machine learning models, providing important insights into the reliability of predictions and aiding the decision-making process in real-world applications. This paper proposes a novel way to use variance-based measures to quantify uncertainty on the basis of second-order distributions in classification problems. A distinctive feature of the measures is the ability to reason about uncertainties on a class-based level, which is useful in situations where nuanced decision-making is required. Recalling some properties from the literature, we highlight that the variance-based measures satisfy important (axiomatic) properties. In addition to this axiomatic approach, we present empirical results showing the measures to be effective and competitive to commonly used entropy-based measures.