Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity
This work addresses the challenge of enhancing neural network adaptability and learning efficiency for AI systems, though it builds incrementally on prior differentiable plasticity research.
The authors tackled the problem of enabling artificial neural networks to exhibit self-modifying, lifelong learning akin to biological brains by introducing a differentiable method for neuromodulated plasticity, resulting in improved performance on reinforcement and supervised learning tasks, such as neuromodulated plastic LSTMs outperforming standard LSTMs on a language modeling benchmark.
The impressive lifelong learning in animal brains is primarily enabled by plastic changes in synaptic connectivity. Importantly, these changes are not passive, but are actively controlled by neuromodulation, which is itself under the control of the brain. The resulting self-modifying abilities of the brain play an important role in learning and adaptation, and are a major basis for biological reinforcement learning. Here we show for the first time that artificial neural networks with such neuromodulated plasticity can be trained with gradient descent. Extending previous work on differentiable Hebbian plasticity, we propose a differentiable formulation for the neuromodulation of plasticity. We show that neuromodulated plasticity improves the performance of neural networks on both reinforcement learning and supervised learning tasks. In one task, neuromodulated plastic LSTMs with millions of parameters outperform standard LSTMs on a benchmark language modeling task (controlling for the number of parameters). We conclude that differentiable neuromodulation of plasticity offers a powerful new framework for training neural networks.