Langevin-gradient parallel tempering for Bayesian neural learning
This work addresses scalability issues in Bayesian neural learning for applications like time series prediction and pattern classification, but it is incremental as it builds on existing MCMC techniques.
The paper tackles convergence and scalability challenges in Bayesian neural networks by combining parallel tempering with Langevin-gradient Metropolis-Hastings proposals, resulting in improved computational time and better prediction or decision-making capabilities compared to related methods.
Bayesian neural learning feature a rigorous approach to estimation and uncertainty quantification via the posterior distribution of weights that represent knowledge of the neural network. This not only provides point estimates of optimal set of weights but also the ability to quantify uncertainty in decision making using the posterior distribution. Markov chain Monte Carlo (MCMC) techniques are typically used to obtain sample-based estimates of the posterior distribution. However, these techniques face challenges in convergence and scalability, particularly in settings with large datasets and network architectures. This paper address these challenges in two ways. First, parallel tempering is used used to explore multiple modes of the posterior distribution and implemented in multi-core computing architecture. Second, we make within-chain sampling schemes more efficient by using Langevin gradient information in forming Metropolis-Hastings proposal distributions. We demonstrate the techniques using time series prediction and pattern classification applications. The results show that the method not only improves the computational time, but provides better prediction or decision making capabilities when compared to related methods.