Hebbian Synaptic Modifications in Spiking Neurons that Learn
This work addresses the problem of developing biologically plausible learning rules for spiking neurons, which could impact computational neuroscience and neuromorphic computing, though it appears incremental as it builds on existing reinforcement learning frameworks.
The authors derived a new synaptic plasticity model from reinforcement learning algorithms, showing it yields locally optimal performance for multi-agent reinforcement learning without explicit communication, and demonstrated its effectiveness in simple pattern classification and motor learning tasks through simulations.
In this paper, we derive a new model of synaptic plasticity, based on recent algorithms for reinforcement learning (in which an agent attempts to learn appropriate actions to maximize its long-term average reward). We show that these direct reinforcement learning algorithms also give locally optimal performance for the problem of reinforcement learning with multiple agents, without any explicit communication between agents. By considering a network of spiking neurons as a collection of agents attempting to maximize the long-term average of a reward signal, we derive a synaptic update rule that is qualitatively similar to Hebb's postulate. This rule requires only simple computations, such as addition and leaky integration, and involves only quantities that are available in the vicinity of the synapse. Furthermore, it leads to synaptic connection strengths that give locally optimal values of the long term average reward. The reinforcement learning paradigm is sufficiently broad to encompass many learning problems that are solved by the brain. We illustrate, with simulations, that the approach is effective for simple pattern classification and motor learning tasks.