Time to Spike? Understanding the Representational Power of Spiking Neural Networks in Discrete Time
This work addresses a foundational gap in understanding SNNs for researchers in neuromorphic computing, though it is incremental as it builds on existing models.
The paper tackled the limited theoretical understanding of spiking neural networks (SNNs) by analyzing discrete-time leaky integrate-and-fire SNNs, demonstrating they realize piecewise constant functions and quantifying network size for approximating continuous functions, with numerical experiments supporting the findings.
Recent years have seen significant progress in developing spiking neural networks (SNNs) as a potential solution to the energy challenges posed by conventional artificial neural networks (ANNs). However, our theoretical understanding of SNNs remains relatively limited compared to the ever-growing body of literature on ANNs. In this paper, we study a discrete-time model of SNNs based on leaky integrate-and-fire (LIF) neurons, referred to as discrete-time LIF-SNNs, a widely used framework that still lacks solid theoretical foundations. We demonstrate that discrete-time LIF-SNNs with static inputs and outputs realize piecewise constant functions defined on polyhedral regions, and more importantly, we quantify the network size required to approximate continuous functions. Moreover, we investigate the impact of latency (number of time steps) and depth (number of layers) on the complexity of the input space partitioning induced by discrete-time LIF-SNNs. Our analysis highlights the importance of latency and contrasts these networks with ANNs employing piecewise linear activation functions. Finally, we present numerical experiments to support our theoretical findings.