Physics-Informed Machine Learning of Dynamical Systems for Efficient Bayesian Inference
This work addresses efficiency bottlenecks in Bayesian inference for practitioners in fields like statistics and machine learning, representing an incremental improvement by adapting existing physics-based methods to a new application.
The paper tackles the computational expense of gradient calculations in Bayesian inference using the no-u-turn sampler (NUTS) by proposing latent variable Hamiltonian neural networks (L-HNNs) integrated with an online error monitoring scheme, achieving up to 50% reduction in gradient evaluations while maintaining sampling accuracy in high-dimensional posterior densities.
Although the no-u-turn sampler (NUTS) is a widely adopted method for performing Bayesian inference, it requires numerous posterior gradients which can be expensive to compute in practice. Recently, there has been a significant interest in physics-based machine learning of dynamical (or Hamiltonian) systems and Hamiltonian neural networks (HNNs) is a noteworthy architecture. But these types of architectures have not been applied to solve Bayesian inference problems efficiently. We propose the use of HNNs for performing Bayesian inference efficiently without requiring numerous posterior gradients. We introduce latent variable outputs to HNNs (L-HNNs) for improved expressivity and reduced integration errors. We integrate L-HNNs in NUTS and further propose an online error monitoring scheme to prevent sampling degeneracy in regions where L-HNNs may have little training data. We demonstrate L-HNNs in NUTS with online error monitoring considering several complex high-dimensional posterior densities and compare its performance to NUTS.