Adaptive Uncertainty Quantification for Generative AI
This work addresses the problem of providing reliable uncertainty estimates for users of black-box generative AI models, offering an incremental improvement over traditional split-conformal methods with adaptive local calibration.
The paper tackles uncertainty quantification for black-box generative AI models by developing a conformal prediction wrapper that adaptively partitions the predictor space and calibrates conformity scores locally, achieving finite sample group-conditional coverage guarantees and demonstrating substantial local tightening of uncertainty sets while maintaining marginal coverage in applications like GPT-4o predictions.
This work is concerned with conformal prediction in contemporary applications (including generative AI) where a black-box model has been trained on data that are not accessible to the user. Mirroring split-conformal inference, we design a wrapper around a black-box algorithm which calibrates conformity scores. This calibration is local and proceeds in two stages by first adaptively partitioning the predictor space into groups and then calibrating sectionally group by group. Adaptive partitioning (self-grouping) is achieved by fitting a robust regression tree to the conformity scores on the calibration set. This new tree variant is designed in such a way that adding a single new observation does not change the tree fit with overwhelmingly large probability. This add-one-in robustness property allows us to conclude a finite sample group-conditional coverage guarantee, a refinement of the marginal guarantee. In addition, unlike traditional split-conformal inference, adaptive splitting and within-group calibration yields adaptive bands which can stretch and shrink locally. We demonstrate benefits of local tightening on several simulated as well as real examples using non-parametric regression. Finally, we consider two contemporary classification applications for obtaining uncertainty quantification around GPT-4o predictions. We conformalize skin disease diagnoses based on self-reported symptoms as well as predicted states of U.S. legislators based on summaries of their ideology. We demonstrate substantial local tightening of the uncertainty sets while attaining similar marginal coverage.