C-Adapter: Adapting Deep Classifiers for Efficient Conformal Prediction Sets
This work addresses the efficiency-accuracy trade-off in conformal prediction for machine learning practitioners, offering an incremental improvement over existing methods like Conformal Training.
The paper tackles the problem of suboptimal efficiency in conformal prediction sets due to accuracy deterioration from regularization, and introduces C-Adapter, an adapter-based tuning method that enhances efficiency without sacrificing accuracy, achieving improved performance across various classifiers and coverage rates.
Conformal prediction, as an emerging uncertainty quantification technique, typically functions as post-hoc processing for the outputs of trained classifiers. To optimize the classifier for maximum predictive efficiency, Conformal Training rectifies the training objective with a regularization that minimizes the average prediction set size at a specific error rate. However, the regularization term inevitably deteriorates the classification accuracy and leads to suboptimal efficiency of conformal predictors. To address this issue, we introduce \textbf{Conformal Adapter} (C-Adapter), an adapter-based tuning method to enhance the efficiency of conformal predictors without sacrificing accuracy. In particular, we implement the adapter as a class of intra order-preserving functions and tune it with our proposed loss that maximizes the discriminability of non-conformity scores between correctly and randomly matched data-label pairs. Using C-Adapter, the model tends to produce extremely high non-conformity scores for incorrect labels, thereby enhancing the efficiency of prediction sets across different coverage rates. Extensive experiments demonstrate that C-Adapter can effectively adapt various classifiers for efficient prediction sets, as well as enhance the conformal training method.