Mapping high-performance RNNs to in-memory neuromorphic chips
This work addresses the problem of low accuracy in spiking neural networks for energy-efficient neuromorphic computing, offering a novel method that is incremental in improving existing spiking models.
The paper tackles the performance gap between spiking and non-spiking neural networks by proposing an adaptive spiking neuron model abstracted as a low-pass filter, which improves inference performance in recurrent neural networks and achieves at least 500x higher energy-efficiency on neuromorphic chips compared to a Cortex-M4 microprocessor.
The increasing need for compact and low-power computing solutions for machine learning applications has triggered significant interest in energy-efficient neuromorphic systems. However, most of these architectures rely on spiking neural networks, which typically perform poorly compared to their non-spiking counterparts in terms of accuracy. In this paper, we propose a new adaptive spiking neuron model that can be abstracted as a low-pass filter. This abstraction enables faster and better training of spiking networks using back-propagation, without simulating spikes. We show that this model dramatically improves the inference performance of a recurrent neural network and validate it with three complex spatio-temporal learning tasks: the temporal addition task, the temporal copying task, and a spoken-phrase recognition task. We estimate at least 500x higher energy-efficiency using our models on compatible neuromorphic chips in comparison to Cortex-M4, a popular embedded microprocessor.