Multidimensional Uncertainty Quantification via Optimal Transport
This work addresses the need for more robust uncertainty quantification in machine learning models, particularly for downstream applications, though it appears incremental as it builds on existing UQ measures with a novel fusion method.
The paper tackles the problem of uncertainty quantification (UQ) by proposing a multidimensional approach that stacks complementary UQ measures into vectors and uses optimal transport to rank predictions, resulting in improved performance for tasks like selective prediction and out-of-distribution detection across synthetic, image, and text data.
Most uncertainty quantification (UQ) approaches provide a single scalar value as a measure of model reliability. However, different uncertainty measures could provide complementary information on the prediction confidence. Even measures targeting the same type of uncertainty (e.g., ensemble-based and density-based measures of epistemic uncertainty) may capture different failure modes. We take a multidimensional view on UQ by stacking complementary UQ measures into a vector. Such vectors are assigned with Monge-Kantorovich ranks produced by an optimal-transport-based ordering method. The prediction is then deemed more uncertain than the other if it has a higher rank. The resulting VecUQ-OT algorithm uses entropy-regularized optimal transport. The transport map is learned on vectors of scores from in-distribution data and, by design, applies to unseen inputs, including out-of-distribution cases, without retraining. Our framework supports flexible non-additive uncertainty fusion (including aleatoric and epistemic components). It yields a robust ordering for downstream tasks such as selective prediction, misclassification detection, out-of-distribution detection, and selective generation. Across synthetic, image, and text data, VecUQ-OT shows high efficiency even when individual measures fail. The code for the method is available at: https://github.com/stat-ml/multidimensional_uncertainty.