Repeated sequential learning increases memory capacity via effective decorrelation in a recurrent neural network
This work addresses memory capacity limitations in recurrent neural networks, offering a novel approach that could enhance learning efficiency in AI systems, though it appears incremental in its application of existing learning principles.
The study tackled the problem of limited memory capacity in neural networks by introducing a simple local learning rule and repeating sequential learning steps, which drastically increased memory capacity through effective decorrelation and the generation of a pseudo-inverse correlation in connectivity.
Memories in neural system are shaped through the interplay of neural and learning dynamics under external inputs. By introducing a simple local learning rule to a neural network, we found that the memory capacity is drastically increased by sequentially repeating the learning steps of input-output mappings. The origin of this enhancement is attributed to the generation of a Psuedo-inverse correlation in the connectivity. This is associated with the emergence of spontaneous activity that intermittently exhibits neural patterns corresponding to embedded memories. Stablization of memories is achieved by a distinct bifurcation from the spontaneous activity under the application of each input.