Dynamical Learning in Deep Asymmetric Recurrent Neural Networks
This work addresses the challenge of efficient learning in AI systems, potentially bridging AI and computational neuroscience, though it appears incremental in its method.
The authors tackled the problem of learning in deep recurrent neural networks without gradient evaluation by introducing asymmetric networks with sparse excitatory couplings, which achieve competitive performance on standard AI benchmarks.
We show that asymmetric deep recurrent neural networks, enhanced with additional sparse excitatory couplings, give rise to an exponentially large, dense accessible manifold of internal representations which can be found by different algorithms, including simple iterative dynamics. Building on the geometrical properties of the stable configurations, we propose a distributed learning scheme in which input-output associations emerge naturally from the recurrent dynamics, without any need of gradient evaluation. A critical feature enabling the learning process is the stability of the configurations reached at convergence, even after removal of the supervisory output signal. Extensive simulations demonstrate that this approach performs competitively on standard AI benchmarks. The model can be generalized in multiple directions, both computational and biological, potentially contributing to narrowing the gap between AI and computational neuroscience.