Is Neuromorphic MNIST neuromorphic? Analyzing the discriminative power of neuromorphic datasets in the time domain
This study questions the utility of existing neuromorphic datasets for the neuromorphic engineering community, revealing they are incremental in advancing SNN research.
The paper analyzed neuromorphic datasets like N-MNIST to assess if they leverage the timing advantages of spiking neural networks (SNNs), finding that ANNs achieved 99.23% accuracy on N-MNIST and 78.01% on N-Caltech101, while an unsupervised SNN reached 91.78% on N-MNIST, indicating that these datasets do not highlight SNNs' unique abilities.
The advantage of spiking neural networks (SNNs) over their predecessors is their ability to spike, enabling them to use spike timing for coding and efficient computing. A neuromorphic dataset should allow a neuromorphic algorithm to clearly show that a SNN is able to perform better on the dataset than an ANN. We have analyzed both N-MNIST and N-Caltech101 along these lines, but focus our study on N-MNIST. First we evaluate if additional information is encoded in the time domain in a neuromoprhic dataset. We show that an ANN trained with backpropagation on frame based versions of N-MNIST and N-Caltech101 images achieve 99.23% and 78.01% accuracy. These are the best classification accuracies obtained on these datasets to date. Second we present the first unsupervised SNN to be trained on N-MNIST and demonstrate results of 91.78%. We also use this SNN for further experiments on N-MNIST to show that rate based SNNs perform better, and precise spike timings are not important in N-MNIST. N-MNIST does not, therefore, highlight the unique ability of SNNs. The conclusion of this study opens an important question in neuromorphic engineering - what, then, constitutes a good neuromorphic dataset?