Simulating Posterior Bayesian Neural Networks with Dependent Weights
This work addresses a theoretical challenge in Bayesian deep learning for researchers, but it appears incremental as it builds on existing Bayesian neural network frameworks.
The paper tackles the problem of simulating posterior Bayesian neural networks with dependent weights by identifying the distribution of the wide width limit for Gaussian likelihoods and providing a sampling algorithm, and it numerically validates these theoretical results, including proving that the conditional output in shallow cases is a Gaussian mixture.
In this paper we consider posterior Bayesian fully connected and feedforward deep neural networks with dependent weights. Particularly, if the likelihood is Gaussian, we identify the distribution of the wide width limit and provide an algorithm to sample from the network. In the shallow case we explicitly compute the distribution of the conditional output, proving that it is a Gaussian mixture. All the theoretical results are numerically validated.