A Hybrid ANN-SNN Architecture for Low-Power and Low-Latency Visual Perception
This work addresses low-power and low-latency visual perception for edge devices, offering a novel hybrid approach that is incremental in combining existing ANN and SNN methods.
The paper tackles the issue of high latency and power consumption in Spiking Neural Networks (SNNs) for visual perception by proposing a hybrid ANN-SNN architecture that initializes SNN states with an auxiliary ANN, resulting in 88% less power consumption with only a 4% performance drop compared to ANNs and a 74% lower error compared to SNNs.
Spiking Neural Networks (SNN) are a class of bio-inspired neural networks that promise to bring low-power and low-latency inference to edge devices through asynchronous and sparse processing. However, being temporal models, SNNs depend heavily on expressive states to generate predictions on par with classical artificial neural networks (ANNs). These states converge only after long transient periods, and quickly decay without input data, leading to higher latency, power consumption, and lower accuracy. This work addresses this issue by initializing the state with an auxiliary ANN running at a low rate. The SNN then uses the state to generate predictions with high temporal resolution until the next initialization phase. Our hybrid ANN-SNN model thus combines the best of both worlds: It does not suffer from long state transients and state decay thanks to the ANN, and can generate predictions with high temporal resolution, low latency, and low power thanks to the SNN. We show for the task of event-based 2D and 3D human pose estimation that our method consumes 88% less power with only a 4% decrease in performance compared to its fully ANN counterparts when run at the same inference rate. Moreover, when compared to SNNs, our method achieves a 74% lower error. This research thus provides a new understanding of how ANNs and SNNs can be used to maximize their respective benefits.