Bayesian Predictive Coding
This work addresses the problem of uncertainty quantification in biologically plausible neural models for researchers in computational neuroscience and deep learning, though it is incremental as it extends an existing framework.
The paper tackles the limitation of predictive coding (PC) in quantifying epistemic uncertainty by introducing Bayesian Predictive Coding (BPC), which estimates a posterior distribution over network parameters. The result shows that BPC converges in fewer epochs in full-batch settings and offers competitive uncertainty quantification compared to existing methods.
Predictive coding (PC) is an influential theory of information processing in the brain, providing a biologically plausible alternative to backpropagation. It is motivated in terms of Bayesian inference, as hidden states and parameters are optimised via gradient descent on variational free energy. However, implementations of PC rely on maximum \textit{a posteriori} (MAP) estimates of hidden states and maximum likelihood (ML) estimates of parameters, limiting their ability to quantify epistemic uncertainty. In this work, we investigate a Bayesian extension to PC that estimates a posterior distribution over network parameters. This approach, termed Bayesian Predictive coding (BPC), preserves the locality of PC and results in closed-form Hebbian weight updates. Compared to PC, our BPC algorithm converges in fewer epochs in the full-batch setting and remains competitive in the mini-batch setting. Additionally, we demonstrate that BPC offers uncertainty quantification comparable to existing methods in Bayesian deep learning, while also improving convergence properties. Together, these results suggest that BPC provides a biologically plausible method for Bayesian learning in the brain, as well as an attractive approach to uncertainty quantification in deep learning.