Valid Selection among Conformal Sets
This addresses a practical challenge in conformal prediction for machine learning practitioners, offering a method to improve set selection while maintaining theoretical guarantees, though it is incremental in nature.
The paper tackles the problem of selecting the smallest valid conformal prediction set without invalidating coverage guarantees, proposing a stability-based approach that ensures coverage and extends to online settings, with experimental validation.
Conformal prediction offers a distribution-free framework for constructing prediction sets with coverage guarantees. In practice, multiple valid conformal prediction sets may be available, arising from different models or methodologies. However, selecting the most desirable set, such as the smallest, can invalidate the coverage guarantees. To address this challenge, we propose a stability-based approach that ensures coverage for the selected prediction set. We extend our results to the online conformal setting, propose several refinements in settings where additional structure is available, and demonstrate its effectiveness through experiments.