Evolving Decomposed Plasticity Rules for Information-Bottlenecked Meta-Learning
This work addresses the genomics bottleneck in meta-learning for AI agents, offering a biologically inspired method to improve generalization in dynamic environments like random mazes.
The paper tackles the problem of meta-learning with plastic neural networks by decomposing connection-dependent plasticity rules into neuron-dependent ones, reducing meta-parameters from Θ(n²) to Θ(n). The results show that this approach enables randomly initialized RNNs to surpass pre-trained models in long-term memorization tasks, with decomposed plasticity being comparable or more effective than canonical rules in some cases.
Artificial neural networks (ANNs) are typically confined to accomplishing pre-defined tasks by learning a set of static parameters. In contrast, biological neural networks (BNNs) can adapt to various new tasks by continually updating the neural connections based on the inputs, which is aligned with the paradigm of learning effective learning rules in addition to static parameters, e.g., meta-learning. Among various biologically inspired learning rules, Hebbian plasticity updates the neural network weights using local signals without the guide of an explicit target function, thus enabling an agent to learn automatically without human efforts. However, typical plastic ANNs using a large amount of meta-parameters violate the nature of the genomics bottleneck and potentially deteriorate the generalization capacity. This work proposes a new learning paradigm decomposing those connection-dependent plasticity rules into neuron-dependent rules thus accommodating $Θ(n^2)$ learnable parameters with only $Θ(n)$ meta-parameters. We also thoroughly study the effect of different neural modulation on plasticity. Our algorithms are tested in challenging random 2D maze environments, where the agents have to use their past experiences to shape the neural connections and improve their performances for the future. The results of our experiment validate the following: 1. Plasticity can be adopted to continually update a randomly initialized RNN to surpass pre-trained, more sophisticated recurrent models, especially when coming to long-term memorization. 2. Following the genomics bottleneck, the proposed decomposed plasticity can be comparable to or even more effective than canonical plasticity rules in some instances.