Leave-One-Out Stable Conformal Prediction
This work addresses computational bottlenecks in distribution-free uncertainty quantification for machine learning practitioners, offering a faster alternative to existing conformal prediction methods.
The authors tackled the challenge of balancing computational efficiency and prediction accuracy in conformal prediction, particularly for multiple predictions, by proposing Leave-One-Out Stable Conformal Prediction (LOO-StabCP), which is much faster than existing methods and demonstrates superior numerical performance on synthetic and real-world data, including improved test power in a screening problem compared to state-of-the-art split conformal methods.
Conformal prediction (CP) is an important tool for distribution-free predictive uncertainty quantification. Yet, a major challenge is to balance computational efficiency and prediction accuracy, particularly for multiple predictions. We propose Leave-One-Out Stable Conformal Prediction (LOO-StabCP), a novel method to speed up full conformal using algorithmic stability without sample splitting. By leveraging leave-one-out stability, our method is much faster in handling a large number of prediction requests compared to existing method RO-StabCP based on replace-one stability. We derived stability bounds for several popular machine learning tools: regularized loss minimization (RLM) and stochastic gradient descent (SGD), as well as kernel method, neural networks and bagging. Our method is theoretically justified and demonstrates superior numerical performance on synthetic and real-world data. We applied our method to a screening problem, where its effective exploitation of training data led to improved test power compared to state-of-the-art method based on split conformal.