General policy mapping: online continual reinforcement learning inspired on the insect brain
This work addresses efficient online reinforcement learning for resource-constrained scenarios, representing an incremental advance in continual learning methods.
The paper tackles the problem of enabling reinforcement learning algorithms to converge in online continual learning settings by developing a model inspired by the insect brain, which uses a shared general policy layer to achieve positive backward transfer and improve performance on older tasks.
We have developed a model for online continual or lifelong reinforcement learning (RL) inspired on the insect brain. Our model leverages the offline training of a feature extraction and a common general policy layer to enable the convergence of RL algorithms in online settings. Sharing a common policy layer across tasks leads to positive backward transfer, where the agent continuously improved in older tasks sharing the same underlying general policy. Biologically inspired restrictions to the agent's network are key for the convergence of RL algorithms. This provides a pathway towards efficient online RL in resource-constrained scenarios.