Test-time augmentation improves efficiency in conformal prediction
This work improves the efficiency of conformal prediction for users needing reliable uncertainty estimates, though it is incremental as it builds on existing methods.
The paper tackled the problem of uninformatively large prediction sets in conformal classifiers by applying test-time augmentation, which reduced set sizes by 10%-14% on average without requiring model retraining.
A conformal classifier produces a set of predicted classes and provides a probabilistic guarantee that the set includes the true class. Unfortunately, it is often the case that conformal classifiers produce uninformatively large sets. In this work, we show that test-time augmentation (TTA)--a technique that introduces inductive biases during inference--reduces the size of the sets produced by conformal classifiers. Our approach is flexible, computationally efficient, and effective. It can be combined with any conformal score, requires no model retraining, and reduces prediction set sizes by 10%-14% on average. We conduct an evaluation of the approach spanning three datasets, three models, two established conformal scoring methods, different guarantee strengths, and several distribution shifts to show when and why test-time augmentation is a useful addition to the conformal pipeline.