Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
This work addresses the challenge of better uncertainty estimation in Bayesian neural networks for machine learning practitioners, representing an incremental advancement in variational inference techniques.
The paper tackles the problem of improving variational approximations for Bayesian neural networks by reinterpreting multiplicative noise as auxiliary variables, enabling the use of normalizing flows while maintaining efficiency. The result is a significant improvement over classical mean field methods in predictive accuracy and uncertainty, as demonstrated in experiments.
We reinterpret multiplicative noise in neural networks as auxiliary random variables that augment the approximate posterior in a variational setting for Bayesian neural networks. We show that through this interpretation it is both efficient and straightforward to improve the approximation by employing normalizing flows while still allowing for local reparametrizations and a tractable lower bound. In experiments we show that with this new approximation we can significantly improve upon classical mean field for Bayesian neural networks on both predictive accuracy as well as predictive uncertainty.