Brain-Inspired Learning on Neuromorphic Substrates
This work addresses the problem of enabling efficient online learning on neuromorphic substrates for scalable, low-power temporal data processing, representing an incremental advancement in bridging deep learning and neuromorphic computing.
The paper tackles the challenge of training neuromorphic hardware by developing a mathematical framework that connects Real-Time Recurrent Learning to biologically plausible rules for Spiking Neural Networks, using a sparse approximation to reduce computational complexity and improve applicability.
Neuromorphic hardware strives to emulate brain-like neural networks and thus holds the promise for scalable, low-power information processing on temporal data streams. Yet, to solve real-world problems, these networks need to be trained. However, training on neuromorphic substrates creates significant challenges due to the offline character and the required non-local computations of gradient-based learning algorithms. This article provides a mathematical framework for the design of practical online learning algorithms for neuromorphic substrates. Specifically, we show a direct connection between Real-Time Recurrent Learning (RTRL), an online algorithm for computing gradients in conventional Recurrent Neural Networks (RNNs), and biologically plausible learning rules for training Spiking Neural Networks (SNNs). Further, we motivate a sparse approximation based on block-diagonal Jacobians, which reduces the algorithm's computational complexity, diminishes the non-local information requirements, and empirically leads to good learning performance, thereby improving its applicability to neuromorphic substrates. In summary, our framework bridges the gap between synaptic plasticity and gradient-based approaches from deep learning and lays the foundations for powerful information processing on future neuromorphic hardware systems.