PAC Confidence Sets for Deep Neural Networks via Calibrated Prediction
This work addresses the need for reliable uncertainty quantification in deep learning, offering a practical solution for safety-critical applications, though it is incremental as it builds on existing calibration and generalization techniques.
The authors tackled the problem of providing provable confidence sets for deep neural networks by combining calibrated prediction with generalization bounds, achieving PAC guarantees that ensure the true label is contained with high probability across tasks like ImageNet classification, visual object tracking, and reinforcement learning dynamics.
We propose an algorithm combining calibrated prediction and generalization bounds from learning theory to construct confidence sets for deep neural networks with PAC guarantees---i.e., the confidence set for a given input contains the true label with high probability. We demonstrate how our approach can be used to construct PAC confidence sets on ResNet for ImageNet, a visual object tracking model, and a dynamics model for the half-cheetah reinforcement learning problem.