Towards Accurate and High-Speed Spiking Neuromorphic Systems with Data Quantization-Aware Deep Networks
This work addresses the problem of accuracy degradation in deploying deep neural networks on spiking neuromorphic systems for researchers and engineers in neuromorphic computing, representing an incremental improvement by focusing on inter-layer signal quantization.
The paper tackles the accuracy loss from quantizing data in deep neural networks deployed on spiking neuromorphic systems by proposing fixed integer inter-layer signals and fixed-point weights, achieving accuracy loss within 0.02% on MNIST and 2.3% on CIFAR-10 with 4-bit representation, along with over 9.8x speedup, 89.1% energy saving, and 30% area saving compared to 8-bit dynamic fixed-point DNNs.
Deep Neural Networks (DNNs) have gained immense success in cognitive applications and greatly pushed today's artificial intelligence forward. The biggest challenge in executing DNNs is their extremely data-extensive computations. The computing efficiency in speed and energy is constrained when traditional computing platforms are employed in such computational hungry executions. Spiking neuromorphic computing (SNC) has been widely investigated in deep networks implementation own to their high efficiency in computation and communication. However, weights and signals of DNNs are required to be quantized when deploying the DNNs on the SNC, which results in unacceptable accuracy loss. %However, the system accuracy is limited by quantizing data directly in deep networks deployment. Previous works mainly focus on weights discretize while inter-layer signals are mainly neglected. In this work, we propose to represent DNNs with fixed integer inter-layer signals and fixed-point weights while holding good accuracy. We implement the proposed DNNs on the memristor-based SNC system as a deployment example. With 4-bit data representation, our results show that the accuracy loss can be controlled within 0.02% (2.3%) on MNIST (CIFAR-10). Compared with the 8-bit dynamic fixed-point DNNs, our system can achieve more than 9.8x speedup, 89.1% energy saving, and 30% area saving.