Uncertainty Quantification for Large-Scale Deep Networks via Post-StoNet Modeling
This addresses uncertainty quantification for users of large-scale deep neural networks, representing an incremental improvement over existing post-hoc calibration methods.
The paper tackles the problem of uncertainty quantification for large-scale deep neural networks by introducing a post-processing approach that feeds the last hidden layer output into a stochastic neural network (StoNet) with sparse penalty training. The method constructs honest confidence intervals with shorter lengths than conformal methods and achieves better calibration than other post-hoc techniques, as demonstrated in comprehensive experiments.
Deep learning has revolutionized modern data science. However, how to accurately quantify the uncertainty of predictions from large-scale deep neural networks (DNNs) remains an unresolved issue. To address this issue, we introduce a novel post-processing approach. This approach feeds the output from the last hidden layer of a pre-trained large-scale DNN model into a stochastic neural network (StoNet), then trains the StoNet with a sparse penalty on a validation dataset and constructs prediction intervals for future observations. We establish a theoretical guarantee for the validity of this approach; in particular, the parameter estimation consistency for the sparse StoNet is essential for the success of this approach. Comprehensive experiments demonstrate that the proposed approach can construct honest confidence intervals with shorter interval lengths compared to conformal methods and achieves better calibration compared to other post-hoc calibration techniques. Additionally, we show that the StoNet formulation provides us with a platform to adapt sparse learning theory and methods from linear models to DNNs.