A Conformal Prediction Score that is Robust to Label Noise
This addresses label noise issues in CP for medical imaging classification, offering a domain-specific incremental improvement.
The paper tackled the problem of calibrating Conformal Prediction (CP) with noisy labels in validation sets by introducing a robust conformal score, achieving a large margin improvement in average prediction set size while maintaining required coverage on medical imaging datasets.
Conformal Prediction (CP) quantifies network uncertainty by building a small prediction set with a pre-defined probability that the correct class is within this set. In this study we tackle the problem of CP calibration based on a validation set with noisy labels. We introduce a conformal score that is robust to label noise. The noise-free conformal score is estimated using the noisy labeled data and the noise level. In the test phase the noise-free score is used to form the prediction set. We applied the proposed algorithm to several standard medical imaging classification datasets. We show that our method outperforms current methods by a large margin, in terms of the average size of the prediction set, while maintaining the required coverage.