Pitfalls of Conformal Predictions for Medical Image Classification
This work highlights critical limitations for practitioners in safety-critical medical applications, making it an incremental cautionary analysis.
The paper tackles the problem of unreliable uncertainty estimation in medical image classification by demonstrating that conformal predictions fail under distributional shifts and have limited value in small-class settings, showing pitfalls in dermatology and histopathology examples.
Reliable uncertainty estimation is one of the major challenges for medical classification tasks. While many approaches have been proposed, recently the statistical framework of conformal predictions has gained a lot of attention, due to its ability to provide provable calibration guarantees. Nonetheless, the application of conformal predictions in safety-critical areas such as medicine comes with pitfalls, limitations and assumptions that practitioners need to be aware of. We demonstrate through examples from dermatology and histopathology that conformal predictions are unreliable under distributional shifts in input and label variables. Additionally, conformal predictions should not be used for selecting predictions to improve accuracy and are not reliable for subsets of the data, such as individual classes or patient attributes. Moreover, in classification settings with a small number of classes, which are common in medical image classification tasks, conformal predictions have limited practical value.