RR-CP: Reliable-Region-Based Conformal Prediction for Trustworthy Medical Image Classification
This work addresses the need for trustworthy AI in clinical decision-making by improving reliability in medical image classification, though it is incremental as it builds on existing conformal prediction methods.
The paper tackles the problem of ensuring that conformal prediction methods in medical image classification achieve a user-specified error rate (e.g., 0.5%) at test time, proposing RR-CP which optimizes prediction set size under this constraint. Experiments on five datasets show RR-CP achieves the error rate significantly more frequently than existing methods while maintaining small prediction sets.
Conformal prediction (CP) generates a set of predictions for a given test sample such that the prediction set almost always contains the true label (e.g., 99.5\% of the time). CP provides comprehensive predictions on possible labels of a given test sample, and the size of the set indicates how certain the predictions are (e.g., a set larger than one is `uncertain'). Such distinct properties of CP enable effective collaborations between human experts and medical AI models, allowing efficient intervention and quality check in clinical decision-making. In this paper, we propose a new method called Reliable-Region-Based Conformal Prediction (RR-CP), which aims to impose a stronger statistical guarantee so that the user-specified error rate (e.g., 0.5\%) can be achieved in the test time, and under this constraint, the size of the prediction set is optimized (to be small). We consider a small prediction set size an important measure only when the user-specified error rate is achieved. Experiments on five public datasets show that our RR-CP performs well: with a reasonably small-sized prediction set, it achieves the user-specified error rate (e.g., 0.5\%) significantly more frequently than exiting CP methods.