Nonparametric Uncertainty Quantification for Single Deterministic Neural Network
This provides a fast, scalable uncertainty quantification method for practitioners using neural networks, though it appears incremental as it builds on established nonparametric techniques.
The paper tackles uncertainty quantification for neural network predictions by proposing a nonparametric method based on Nadaraya-Watson estimates that explicitly disentangles aleatoric and epistemic uncertainties, demonstrating strong performance on text classification and multiple image datasets including MNIST, SVHN, CIFAR-100, and ImageNet.
This paper proposes a fast and scalable method for uncertainty quantification of machine learning models' predictions. First, we show the principled way to measure the uncertainty of predictions for a classifier based on Nadaraya-Watson's nonparametric estimate of the conditional label distribution. Importantly, the proposed approach allows to disentangle explicitly aleatoric and epistemic uncertainties. The resulting method works directly in the feature space. However, one can apply it to any neural network by considering an embedding of the data induced by the network. We demonstrate the strong performance of the method in uncertainty estimation tasks on text classification problems and a variety of real-world image datasets, such as MNIST, SVHN, CIFAR-100 and several versions of ImageNet.