Transductive conformal inference with adaptive scores
This work addresses the need for robust uncertainty quantification in machine learning tasks like transfer learning and novelty detection, offering theoretical advancements that are incremental but enhance existing conformal inference methods.
The paper tackled the problem of providing distribution-free guarantees for transductive conformal inference with adaptive scores, establishing that the joint distribution of conformal p-values follows a Pólya urn model and proving a concentration inequality for their empirical distribution function.
Conformal inference is a fundamental and versatile tool that provides distribution-free guarantees for many machine learning tasks. We consider the transductive setting, where decisions are made on a test sample of $m$ new points, giving rise to $m$ conformal $p$-values. While classical results only concern their marginal distribution, we show that their joint distribution follows a Pólya urn model, and establish a concentration inequality for their empirical distribution function. The results hold for arbitrary exchangeable scores, including adaptive ones that can use the covariates of the test+calibration samples at training stage for increased accuracy. We demonstrate the usefulness of these theoretical results through uniform, in-probability guarantees for two machine learning tasks of current interest: interval prediction for transductive transfer learning and novelty detection based on two-class classification.