Semi-Supervised Conformal Prediction With Unlabeled Nonconformity Score
This work addresses the challenge of coverage deviation and large prediction sets in conformal prediction for applications with scarce labeled data, representing an incremental improvement.
The paper tackles the problem of uncertainty quantification with limited labeled data in conformal prediction by proposing SemiCP, a semi-supervised method that leverages unlabeled data for calibration, resulting in reduced instability and inefficiency in prediction sets.
Conformal prediction (CP) is a powerful framework for uncertainty quantification, providing prediction sets with coverage guarantees when calibrated on sufficient labeled data. However, in real-world applications where labeled data is often limited, standard CP can lead to coverage deviation and output overly large prediction sets. In this paper, we extend CP to the semi-supervised setting and propose SemiCP, leveraging both labeled data and unlabeled data for calibration. Specifically, we introduce a novel nonconformity score function, NNM, designed for unlabeled data. This function selects labeled data with similar pseudo-label scores to estimate nonconformity scores, integrating them into the calibration process to overcome sample size limitations. We theoretically demonstrate that, under mild assumptions, SemiCP provide asymptotically coverage guarantee for prediction sets. Extensive experiments further validate that our approach effectively reduces instability and inefficiency under limited calibration data, can be adapted to conditional coverage settings, and integrates seamlessly with existing CP methods.