Augmenting Neural Networks with Priors on Function Values
This work solves the problem of leveraging domain-specific prior knowledge in neural networks for researchers in natural sciences, though it is incremental as it builds on existing BNN frameworks.
The paper addresses the challenge of incorporating prior knowledge on function values into Bayesian neural networks (BNNs) to improve accuracy in label-limited settings, particularly in natural sciences like biophysics and quantum chemistry, by developing a probabilistic approach that balances prior information and neural network predictions based on epistemic uncertainty.
The need for function estimation in label-limited settings is common in the natural sciences. At the same time, prior knowledge of function values is often available in these domains. For example, data-free biophysics-based models can be informative on protein properties, while quantum-based computations can be informative on small molecule properties. How can we coherently leverage such prior knowledge to help improve a neural network model that is quite accurate in some regions of input space -- typically near the training data -- but wildly wrong in other regions? Bayesian neural networks (BNN) enable the user to specify prior information only on the neural network weights, not directly on the function values. Moreover, there is in general no clear mapping between these. Herein, we tackle this problem by developing an approach to augment BNNs with prior information on the function values themselves. Our probabilistic approach yields predictions that rely more heavily on the prior information when the epistemic uncertainty is large, and more heavily on the neural network when the epistemic uncertainty is small.