Aggregating Conformal Prediction Sets via α-Allocation
This work addresses a major open problem in conformal prediction for improving efficiency in machine learning uncertainty quantification, though it is incremental as it builds on existing frameworks.
The paper tackled the challenge of reducing prediction set size in conformal prediction by introducing COLA, a strategy that optimally allocates confidence levels across multiple sets, achieving smaller sets than state-of-the-art baselines while maintaining valid coverage.
Conformal prediction offers a distribution-free framework for constructing prediction sets with finite-sample coverage. Yet, efficiently leveraging multiple conformity scores to reduce prediction set size remains a major open challenge. Instead of selecting a single best score, this work introduces a principled aggregation strategy, COnfidence-Level Allocation (COLA), that optimally allocates confidence levels across multiple conformal prediction sets to minimize empirical set size while maintaining provable coverage. Two variants are further developed, COLA-s and COLA-f, which guarantee finite-sample marginal coverage via sample splitting and full conformalization, respectively. In addition, we develop COLA-l, an individualized allocation strategy that promotes local size efficiency while achieving asymptotic conditional coverage. Extensive experiments on synthetic and real-world datasets demonstrate that COLA achieves considerably smaller prediction sets than state-of-the-art baselines while maintaining valid coverage.