Evidential Uncertainty Sets in Deep Classifiers Using Conformal Prediction
This work addresses uncertainty quantification in deep learning for image classification, offering an incremental improvement over existing conformal prediction methods.
The paper tackles the problem of generating conformal prediction sets for image classifiers by proposing Evidential Conformal Prediction (ECP), which uses evidential deep learning to quantify uncertainty and outperforms state-of-the-art methods in set sizes and adaptivity while maintaining coverage.
In this paper, we propose Evidential Conformal Prediction (ECP) method for image classifiers to generate the conformal prediction sets. Our method is designed based on a non-conformity score function that has its roots in Evidential Deep Learning (EDL) as a method of quantifying model (epistemic) uncertainty in DNN classifiers. We use evidence that are derived from the logit values of target labels to compute the components of our non-conformity score function: the heuristic notion of uncertainty in CP, uncertainty surprisal, and expected utility. Our extensive experimental evaluation demonstrates that ECP outperforms three state-of-the-art methods for generating CP sets, in terms of their set sizes and adaptivity while maintaining the coverage of true labels.