Variational Neural Networks
This addresses uncertainty estimation for neural network users, though it appears incremental as it builds on existing Bayesian approaches.
The paper tackles uncertainty estimation in neural networks by proposing a method that samples layer outputs from Gaussian distributions parameterized by mean and variance sub-layers, achieving better uncertainty quality than existing single-bin Bayesian Model Averaging methods like Monte Carlo Dropout or Bayes By Backpropagation.
Bayesian Neural Networks (BNNs) provide a tool to estimate the uncertainty of a neural network by considering a distribution over weights and sampling different models for each input. In this paper, we propose a method for uncertainty estimation in neural networks which, instead of considering a distribution over weights, samples outputs of each layer from a corresponding Gaussian distribution, parametrized by the predictions of mean and variance sub-layers. In uncertainty quality estimation experiments, we show that the proposed method achieves better uncertainty quality than other single-bin Bayesian Model Averaging methods, such as Monte Carlo Dropout or Bayes By Backpropagation methods.