Reliable Prediction Intervals with Directly Optimized Inductive Conformal Regression for Deep Learning
This addresses the need for reliable uncertainty quantification in deep learning regression, offering a method that improves prediction interval quality for applications requiring controlled risk, though it is incremental as it builds on existing inductive conformal predictors.
The paper tackles the problem of generating narrow prediction intervals in deep learning regression while ensuring they cover a preset proportion of real labels, and introduces DOICR, which directly optimizes interval width under validity constraints, outperforming state-of-the-art methods on tabular and image data benchmarks.
By generating prediction intervals (PIs) to quantify the uncertainty of each prediction in deep learning regression, the risk of wrong predictions can be effectively controlled. High-quality PIs need to be as narrow as possible, whilst covering a preset proportion of real labels. At present, many approaches to improve the quality of PIs can effectively reduce the width of PIs, but they do not ensure that enough real labels are captured. Inductive Conformal Predictor (ICP) is an algorithm that can generate effective PIs which is theoretically guaranteed to cover a preset proportion of data. However, typically ICP is not directly optimized to yield minimal PI width. However, in this study, we use Directly Optimized Inductive Conformal Regression (DOICR) that takes only the average width of PIs as the loss function and increases the quality of PIs through an optimized scheme under the validity condition that sufficient real labels are captured in the PIs. Benchmark experiments show that DOICR outperforms current state-of-the-art algorithms for regression problems using underlying Deep Neural Network structures for both tabular and image data.