Towards biologically plausible Dreaming and Planning in recurrent spiking networks
This work addresses the challenge of slow learning in AI for applications requiring efficient skill acquisition, though it is incremental in combining existing concepts like dreaming with spiking networks.
The paper tackles the problem of data-inefficient reinforcement learning by proposing a biologically plausible spiking neural network that uses dreaming and planning to boost learning, achieving significant performance improvements without detailed experience storage.
Humans and animals can learn new skills after practicing for a few hours, while current reinforcement learning algorithms require a large amount of data to achieve good performances. Recent model-based approaches show promising results by reducing the number of necessary interactions with the environment to learn a desirable policy. However, these methods require biological implausible ingredients, such as the detailed storage of older experiences, and long periods of offline learning. The optimal way to learn and exploit word-models is still an open question. Taking inspiration from biology, we suggest that dreaming might be an efficient expedient to use an inner model. We propose a two-module (agent and model) spiking neural network in which "dreaming" (living new experiences in a model-based simulated environment) significantly boosts learning. We also explore "planning", an online alternative to dreaming, that shows comparable performances. Importantly, our model does not require the detailed storage of experiences, and learns online the world-model and the policy. Moreover, we stress that our network is composed of spiking neurons, further increasing the biological plausibility and implementability in neuromorphic hardware.