Is Learning in Biological Neural Networks based on Stochastic Gradient Descent? An analysis using stochastic processes
This addresses a fundamental debate in neuroscience and machine learning about learning mechanisms in the brain, with potential implications for understanding biological intelligence and designing brain-inspired algorithms.
The paper tackles the problem of whether learning in biological neural networks (BNNs) uses stochastic gradient descent by analyzing a stochastic model for supervised learning, showing that a continuous gradient step approximately occurs when many local updates process each learning opportunity, suggesting stochastic gradient descent may optimize BNNs.
In recent years, there has been an intense debate about how learning in biological neural networks (BNNs) differs from learning in artificial neural networks. It is often argued that the updating of connections in the brain relies only on local information, and therefore a stochastic gradient-descent type optimization method cannot be used. In this paper, we study a stochastic model for supervised learning in BNNs. We show that a (continuous) gradient step occurs approximately when each learning opportunity is processed by many local updates. This result suggests that stochastic gradient descent may indeed play a role in optimizing BNNs.