Beyond Conformal Predictors: Adaptive Conformal Inference with Confidence Predictors
This provides a more flexible and efficient alternative for uncertainty quantification in machine learning under non-exchangeability, though it is incremental as it builds on existing ACI methods.
The study tackled the problem of ensuring finite-sample coverage guarantees in non-exchangeable data by showing that Adaptive Conformal Inference (ACI) works with Non-Conformal Confidence Predictors (NCCP), not just Conformal Predictors (CP). The result demonstrated that NCCP offers computational advantages in online settings and improved efficiency in batch settings, particularly with limited data.
Adaptive Conformal Inference (ACI) provides finite-sample coverage guarantees, enhancing the prediction reliability under non-exchangeability. This study demonstrates that these desirable properties of ACI do not require the use of Conformal Predictors (CP). We show that the guarantees hold for the broader class of confidence predictors, defined by the requirement of producing nested prediction sets, a property we argue is essential for meaningful confidence statements. We empirically investigate the performance of Non-Conformal Confidence Predictors (NCCP) against CP when used with ACI on non-exchangeable data. In online settings, the NCCP offers significant computational advantages while maintaining a comparable predictive efficiency. In batch settings, inductive NCCP (INCCP) can outperform inductive CP (ICP) by utilising the full training dataset without requiring a separate calibration set, leading to improved efficiency, particularly when the data are limited. Although these initial results highlight NCCP as a theoretically sound and practically effective alternative to CP for uncertainty quantification with ACI in non-exchangeable scenarios, further empirical studies are warranted across diverse datasets and predictors.