Stochastic Bayesian Neural Networks
This work addresses computational bottlenecks in Bayesian neural networks for researchers and practitioners in machine learning, though it appears incremental as it builds on existing variational inference techniques.
The paper tackles the challenge of calculating posterior distributions in Bayesian neural networks by introducing a stochastic Bayesian neural network that maximizes a new Stochastic Evidence Lower Bound objective. The method outperforms previous state-of-the-art algorithms on 5 UCI datasets, showing improved test RMSE and log likelihood while being scalable to larger datasets.
Bayesian neural networks perform variational inference over the weights however calculation of the posterior distribution remains a challenge. Our work builds on variational inference techniques for bayesian neural networks using the original Evidence Lower Bound. In this paper, we present a stochastic bayesian neural network in which we maximize Evidence Lower Bound using a new objective function which we name as Stochastic Evidence Lower Bound. We evaluate our network on 5 publicly available UCI datasets using test RMSE and log likelihood as the evaluation metrics. We demonstrate that our work not only beats the previous state of the art algorithms but is also scalable to larger datasets.