Learning Credal Ensembles via Distributionally Robust Optimization
This work addresses the need for more robust uncertainty quantification in machine learning models, particularly for applications like medical diagnosis, though it is incremental in refining existing credal methods.
The paper tackled the problem of defining and capturing epistemic uncertainty beyond just training randomness by proposing CreDRO, which learns ensembles via distributionally robust optimization to account for potential distribution shifts, resulting in consistent outperformance over existing methods in tasks like out-of-distribution detection and selective classification.
Credal predictors are models that are aware of epistemic uncertainty and produce a convex set of probabilistic predictions. They offer a principled way to quantify predictive epistemic uncertainty (EU) and have been shown to improve model robustness in various settings. However, most state-of-the-art methods mainly define EU as disagreement caused by random training initializations, which mostly reflects sensitivity to optimization randomness rather than uncertainty from deeper sources. To address this, we define EU as disagreement among models trained with varying relaxations of the i.i.d. assumption between training and test data. Based on this idea, we propose CreDRO, which learns an ensemble of plausible models through distributionally robust optimization. As a result, CreDRO captures EU not only from training randomness but also from meaningful disagreement due to potential distribution shifts between training and test data. Empirical results show that CreDRO consistently outperforms existing credal methods on tasks such as out-of-distribution detection across multiple benchmarks and selective classification in medical applications.