Conformal Prediction with Partially Labeled Data
This work addresses the challenge of uncertainty quantification in machine learning when training data is imprecise, which is incremental as it extends conformal prediction to a new data setting.
The paper tackles the problem of applying conformal prediction to set-valued training data, a scenario common in weakly supervised learning, and proposes a generalized method that proves valid and shows favorable performance compared to baselines in experiments.
While the predictions produced by conformal prediction are set-valued, the data used for training and calibration is supposed to be precise. In the setting of superset learning or learning from partial labels, a variant of weakly supervised learning, it is exactly the other way around: training data is possibly imprecise (set-valued), but the model induced from this data yields precise predictions. In this paper, we combine the two settings by making conformal prediction amenable to set-valued training data. We propose a generalization of the conformal prediction procedure that can be applied to set-valued training and calibration data. We prove the validity of the proposed method and present experimental studies in which it compares favorably to natural baselines.