Learning Continuous Chaotic Attractors with a Reservoir Computer
This work addresses the problem of understanding abstraction mechanisms in neural systems for advancing artificial RNN design and neural basis studies, though it is incremental in explaining known bottlenecks.
The researchers trained a 1000-neuron reservoir computer to abstract continuous dynamical attractor memories from isolated examples, such as stable limit cycles or chaotic Lorenz attractors, resulting in a continuum of attractors quantified by an extra Lyapunov exponent equal to zero.
Neural systems are well known for their ability to learn and store information as memories. Even more impressive is their ability to abstract these memories to create complex internal representations, enabling advanced functions such as the spatial manipulation of mental representations. While recurrent neural networks (RNNs) are capable of representing complex information, the exact mechanisms of how dynamical neural systems perform abstraction are still not well-understood, thereby hindering the development of more advanced functions. Here, we train a 1000-neuron RNN -- a reservoir computer (RC) -- to abstract a continuous dynamical attractor memory from isolated examples of dynamical attractor memories. Further, we explain the abstraction mechanism with new theory. By training the RC on isolated and shifted examples of either stable limit cycles or chaotic Lorenz attractors, the RC learns a continuum of attractors, as quantified by an extra Lyapunov exponent equal to zero. We propose a theoretical mechanism of this abstraction by combining ideas from differentiable generalized synchronization and feedback dynamics. Our results quantify abstraction in simple neural systems, enabling us to design artificial RNNs for abstraction, and leading us towards a neural basis of abstraction.