Backpropagation and Biological Plausibility
This addresses the long-standing issue of biological plausibility in neural computation for the machine learning and neuroscience communities, potentially enabling novel learning algorithms with architectural optimization.
The paper tackles the biological plausibility of backpropagation by framing supervised learning in a Lagrangian framework, resulting in the emergence of biologically plausible local algorithms based on saddle point searches in an adjoint space.
By and large, Backpropagation (BP) is regarded as one of the most important neural computation algorithms at the basis of the progress in machine learning, including the recent advances in deep learning. However, its computational structure has been the source of many debates on its arguable biological plausibility. In this paper, it is shown that when framing supervised learning in the Lagrangian framework, while one can see a natural emergence of Backpropagation, biologically plausible local algorithms can also be devised that are based on the search for saddle points in the learning adjoint space composed of weights, neural outputs, and Lagrangian multipliers. This might open the doors to a truly novel class of learning algorithms where, because of the introduction of the notion of support neurons, the optimization scheme also plays a fundamental role in the construction of the architecture.