What is Flagged in Uncertainty Quantification? Latent Density Models for Uncertainty Categorization
This work addresses the need for clearer interpretation of uncertainty flags in machine learning, which is crucial for building trustworthy models, though it is incremental as it builds on existing UQ methods.
The authors tackled the problem of interpreting what uncertainty quantification (UQ) methods flag as suspicious in classification tasks by proposing a framework that categorizes uncertain examples into out-of-distribution, boundary, and in-distribution misclassification classes using a confusion density matrix. They demonstrated through experiments that this framework offers a distinct perspective for comparing UQ methods, serving as a valuable benchmark.
Uncertainty Quantification (UQ) is essential for creating trustworthy machine learning models. Recent years have seen a steep rise in UQ methods that can flag suspicious examples, however, it is often unclear what exactly these methods identify. In this work, we propose a framework for categorizing uncertain examples flagged by UQ methods in classification tasks. We introduce the confusion density matrix -- a kernel-based approximation of the misclassification density -- and use this to categorize suspicious examples identified by a given uncertainty method into three classes: out-of-distribution (OOD) examples, boundary (Bnd) examples, and examples in regions of high in-distribution misclassification (IDM). Through extensive experiments, we show that our framework provides a new and distinct perspective for assessing differences between uncertainty quantification methods, thereby forming a valuable assessment benchmark.