On Feynman--Kac training of partial Bayesian neural networks
This work addresses the training difficulty of pBNNs, which are computationally efficient alternatives to full Bayesian neural networks, offering improved predictive performance for machine learning practitioners.
The authors tackled the challenge of training partial Bayesian neural networks (pBNNs), which are multi-modal and hard to approximate, by proposing a sampling-based training strategy using Feynman--Kac models and sequential Monte Carlo samplers. They demonstrated that this approach outperforms state-of-the-art methods in predictive performance on synthetic and real-world datasets.
Recently, partial Bayesian neural networks (pBNNs), which only consider a subset of the parameters to be stochastic, were shown to perform competitively with full Bayesian neural networks. However, pBNNs are often multi-modal in the latent variable space and thus challenging to approximate with parametric models. To address this problem, we propose an efficient sampling-based training strategy, wherein the training of a pBNN is formulated as simulating a Feynman--Kac model. We then describe variations of sequential Monte Carlo samplers that allow us to simultaneously estimate the parameters and the latent posterior distribution of this model at a tractable computational cost. Using various synthetic and real-world datasets we show that our proposed training scheme outperforms the state of the art in terms of predictive performance.