Meta-Reinforcement Learning with Self-Modifying Networks
This addresses the need for more efficient and generic learning in AI, offering a novel approach inspired by biological plasticity, though it appears incremental in combining meta-learning with dynamic networks.
The paper tackles the problem of slow, specialized learning in deep reinforcement learning by proposing networks with dynamic weights that self-modify based on synaptic state and feedback, resulting in a system capable of one-shot learning, generalization to unseen environments, and adaptive motor policies.
Deep Reinforcement Learning has demonstrated the potential of neural networks tuned with gradient descent for solving complex tasks in well-delimited environments. However, these neural systems are slow learners producing specialized agents with no mechanism to continue learning beyond their training curriculum. On the contrary, biological synaptic plasticity is persistent and manifold, and has been hypothesized to play a key role in executive functions such as working memory and cognitive flexibility, potentially supporting more efficient and generic learning abilities. Inspired by this, we propose to build networks with dynamic weights, able to continually perform self-reflexive modification as a function of their current synaptic state and action-reward feedback, rather than a fixed network configuration. The resulting model, MetODS (for Meta-Optimized Dynamical Synapses) is a broadly applicable meta-reinforcement learning system able to learn efficient and powerful control rules in the agent policy space. A single layer with dynamic synapses can perform one-shot learning, generalizes navigation principles to unseen environments and manifests a strong ability to learn adaptive motor policies.