BARNN: A Bayesian Autoregressive and Recurrent Neural Network
This provides a principled uncertainty framework for scientific applications like PDE solving and molecular generation, though it is incremental as it builds on variational dropout and prior methods.
The authors tackled the lack of uncertainty quantification in autoregressive and recurrent networks by introducing BARNN, a variational Bayesian framework that turns existing models into Bayesian versions, achieving comparable or superior accuracy in PDE modeling and molecular generation while excelling in uncertainty quantification.
Autoregressive and recurrent networks have achieved remarkable progress across various fields, from weather forecasting to molecular generation and Large Language Models. Despite their strong predictive capabilities, these models lack a rigorous framework for addressing uncertainty, which is key in scientific applications such as PDE solving, molecular generation and Machine Learning Force Fields. To address this shortcoming we present BARNN: a variational Bayesian Autoregressive and Recurrent Neural Network. BARNNs aim to provide a principled way to turn any autoregressive or recurrent model into its Bayesian version. BARNN is based on the variational dropout method, allowing to apply it to large recurrent neural networks as well. We also introduce a temporal version of the "Variational Mixtures of Posteriors" prior (tVAMP-prior) to make Bayesian inference efficient and well-calibrated. Extensive experiments on PDE modelling and molecular generation demonstrate that BARNN not only achieves comparable or superior accuracy compared to existing methods, but also excels in uncertainty quantification and modelling long-range dependencies.