Bayesian Conformal Prediction as a Decision Risk Problem
This work addresses the need for robust uncertainty quantification in machine learning, particularly for practitioners dealing with model misspecification, though it is incremental as it builds on existing split conformal frameworks.
The paper tackles the problem of constructing reliable prediction sets with valid coverage guarantees under model misspecification, using Bayesian conformal prediction (BCP) to achieve comparable set sizes to split conformal prediction while reducing run-to-run variability, such as achieving 81% empirical coverage vs. 49% for Bayesian credible intervals in sparse regression at 80% nominal coverage.
Bayesian posterior predictive densities as non-conformity scores and Bayesian quadrature are used to estimate and minimise the expected prediction set size. Operating within a split conformal framework, BCP provides valid coverage guarantees and demonstrates reliable empirical coverage under model misspecification. Across regression and classification tasks, including distribution-shifted settings such as ImageNet-A, BCP yields prediction sets of comparable size to split conformal prediction, while exhibiting substantially lower run-to-run variability in set size. In sparse regression with nominal coverage of 80 percent, BCP achieves 81 percent empirical coverage under a misspecified prior, whereas Bayesian credible intervals under-cover at 49 percent.