Uncertainty Quantification via Stable Distribution Propagation
This work addresses uncertainty quantification for neural networks, which is crucial for improving reliability in applications like out-of-distribution detection, but it appears incremental as it builds on existing propagation techniques with a specific optimization for ReLU.
The paper tackles the problem of quantifying uncertainty in neural network outputs by proposing a method to propagate stable probability distributions through networks using local linearization, which is shown to be optimal for ReLU non-linearities, and demonstrates its utility in predicting calibrated confidence intervals and selective prediction on out-of-distribution data, with advantages over methods like moment matching.
We propose a new approach for propagating stable probability distributions through neural networks. Our method is based on local linearization, which we show to be an optimal approximation in terms of total variation distance for the ReLU non-linearity. This allows propagating Gaussian and Cauchy input uncertainties through neural networks to quantify their output uncertainties. To demonstrate the utility of propagating distributions, we apply the proposed method to predicting calibrated confidence intervals and selective prediction on out-of-distribution data. The results demonstrate a broad applicability of propagating distributions and show the advantages of our method over other approaches such as moment matching.