The Hidden Uncertainty in a Neural Networks Activations
This work provides a computationally efficient method for quantifying epistemic and aleatoric uncertainty from latent representations, which is significant for practitioners needing to assess model generalization without modifying training or incurring high computational costs.
This paper explores whether the distribution of a neural network's latent representations correlates with epistemic uncertainty, finding that deeper layers provide uncertainty estimates comparable to more computationally expensive methods like deep ensembles. They also show that an additional regularizing loss improves out-of-distribution detection for epistemic uncertainty, though it can lead to ambiguous calibration near the data distribution.
The distribution of a neural network's latent representations has been successfully used to detect out-of-distribution (OOD) data. This work investigates whether this distribution moreover correlates with a model's epistemic uncertainty, thus indicates its ability to generalise to novel inputs. We first empirically verify that epistemic uncertainty can be identified with the surprise, thus the negative log-likelihood, of observing a particular latent representation. Moreover, we demonstrate that the output-conditional distribution of hidden representations also allows quantifying aleatoric uncertainty via the entropy of the predictive distribution. We analyse epistemic and aleatoric uncertainty inferred from the representations of different layers and conclude that deeper layers lead to uncertainty with similar behaviour as established - but computationally more expensive - methods (e.g. deep ensembles). While our approach does not require modifying the training process, we follow prior work and experiment with an additional regularising loss that increases the information in the latent representations. We find that this leads to improved OOD detection of epistemic uncertainty at the cost of ambiguous calibration close to the data distribution. We verify our findings on both classification and regression models.