Conformalized Deep Splines for Optimal and Efficient Prediction Sets
This work addresses uncertainty estimation for high-stakes applications, offering an incremental improvement with novel method components.
The paper tackles uncertainty estimation in machine learning by introducing SPICE, a conformal regression method using neural-network-parameterized splines, which achieves up to 50% reduction in average prediction set sizes and improves conditional coverage compared to baselines.
Uncertainty estimation is critical in high-stakes machine learning applications. One effective way to estimate uncertainty is conformal prediction, which can provide predictive inference with statistical coverage guarantees. We present a new conformal regression method, Spline Prediction Intervals via Conformal Estimation (SPICE), that estimates the conditional density using neural-network-parameterized splines. We prove universal approximation and optimality results for SPICE, which are empirically validated by our experiments. SPICE is compatible with two different efficient-to-compute conformal scores, one oracle-optimal for marginal coverage (SPICE-ND) and the other asymptotically optimal for conditional coverage (SPICE-HPD). Results on benchmark datasets demonstrate SPICE-ND models achieve the smallest average prediction set sizes, including average size reductions of nearly 50% for some datasets compared to the next best baseline. SPICE-HPD models achieve the best conditional coverage compared to baselines. The SPICE implementation is made available.