Projected BNNs: Avoiding weight-space pathologies by learning latent representations of neural network weights
This addresses the problem of uncertainty quantification for high-stakes decisions in machine learning, offering a novel approach to avoid weight-space pathologies.
The paper tackles the challenge of quantifying uncertainty in high-dimensional neural network parameters by introducing a variational inference framework that encodes complex distributions in a low-dimensional latent space, showing improved uncertainty characterization and model generalization across synthetic and real-world datasets.
As machine learning systems get widely adopted for high-stake decisions, quantifying uncertainty over predictions becomes crucial. While modern neural networks are making remarkable gains in terms of predictive accuracy, characterizing uncertainty over the parameters of these models is challenging because of the high dimensionality and complex correlations of the network parameter space. This paper introduces a novel variational inference framework for Bayesian neural networks that (1) encodes complex distributions in high-dimensional parameter space with representations in a low-dimensional latent space, and (2) performs inference efficiently on the low-dimensional representations. Across a large array of synthetic and real-world datasets, we show that our method improves uncertainty characterization and model generalization when compared with methods that work directly in the parameter space.