Deep Directly-Trained Spiking Neural Networks for Object Detection
This work addresses the problem of energy-efficient object detection for applications like robotics and embedded systems, representing a novel advancement rather than an incremental improvement.
The authors tackled the challenge of designing a directly-trained spiking neural network (SNN) for object detection, a regression task, and proposed EMS-YOLO, which outperforms state-of-the-art ANN-SNN conversion methods by achieving comparable performance with only 4 time steps instead of at least 500, while consuming 5.83 times less energy on datasets like COCO and Gen1.
Spiking neural networks (SNNs) are brain-inspired energy-efficient models that encode information in spatiotemporal dynamics. Recently, deep SNNs trained directly have shown great success in achieving high performance on classification tasks with very few time steps. However, how to design a directly-trained SNN for the regression task of object detection still remains a challenging problem. To address this problem, we propose EMS-YOLO, a novel directly-trained SNN framework for object detection, which is the first trial to train a deep SNN with surrogate gradients for object detection rather than ANN-SNN conversion strategies. Specifically, we design a full-spike residual block, EMS-ResNet, which can effectively extend the depth of the directly-trained SNN with low power consumption. Furthermore, we theoretically analyze and prove the EMS-ResNet could avoid gradient vanishing or exploding. The results demonstrate that our approach outperforms the state-of-the-art ANN-SNN conversion methods (at least 500 time steps) in extremely fewer time steps (only 4 time steps). It is shown that our model could achieve comparable performance to the ANN with the same architecture while consuming 5.83 times less energy on the frame-based COCO Dataset and the event-based Gen1 Dataset.