Depth Uncertainty in Neural Networks
This work addresses the need for efficient uncertainty estimation in deep learning for applications with limited computational resources, representing an incremental improvement over existing methods.
The paper tackles the problem of estimating uncertainty in deep neural networks without requiring multiple forward passes, which is computationally expensive. The result is a method that provides uncertainty calibration, robustness to dataset shift, and competitive accuracies with more expensive baselines, validated on regression and image classification tasks.
Existing methods for estimating uncertainty in deep learning tend to require multiple forward passes, making them unsuitable for applications where computational resources are limited. To solve this, we perform probabilistic reasoning over the depth of neural networks. Different depths correspond to subnetworks which share weights and whose predictions are combined via marginalisation, yielding model uncertainty. By exploiting the sequential structure of feed-forward networks, we are able to both evaluate our training objective and make predictions with a single forward pass. We validate our approach on real-world regression and image classification tasks. Our approach provides uncertainty calibration, robustness to dataset shift, and accuracies competitive with more computationally expensive baselines.