Evolving-to-Learn Reinforcement Learning Tasks with Spiking Neural Networks
This work addresses the problem of reducing manual effort in adapting spiking neural networks for neuromorphic hardware, though it is incremental as it builds on existing evolutionary and plasticity methods.
The paper tackles the challenge of manually fine-tuning synaptic plasticity rules for spiking neural networks in reinforcement learning tasks by using an evolutionary algorithm to automatically discover optimal rules, resulting in new rules that outperform baselines on XOR and cart-pole tasks.
Inspired by the natural nervous system, synaptic plasticity rules are applied to train spiking neural networks with local information, making them suitable for online learning on neuromorphic hardware. However, when such rules are implemented to learn different new tasks, they usually require a significant amount of work on task-dependent fine-tuning. This paper aims to make this process easier by employing an evolutionary algorithm that evolves suitable synaptic plasticity rules for the task at hand. More specifically, we provide a set of various local signals, a set of mathematical operators, and a global reward signal, after which a Cartesian genetic programming process finds an optimal learning rule from these components. Using this approach, we find learning rules that successfully solve an XOR and cart-pole task, and discover new learning rules that outperform the baseline rules from literature.