Sparse Convolutional Recurrent Learning for Efficient Event-based Neuromorphic Object Detection
This work addresses the problem of integrating compute-intensive event-based object detection into resource-constrained edge applications, representing an incremental improvement in efficiency for neuromorphic processing.
The paper tackles the challenge of efficient object detection with event cameras for automotive and robotics by proposing SEED, which uses sparse convolutional recurrent learning to achieve over 92% activation sparsity, reducing synaptic operations while maintaining or improving mAP compared to state-of-the-art methods.
Leveraging the high temporal resolution and dynamic range, object detection with event cameras can enhance the performance and safety of automotive and robotics applications in real-world scenarios. However, processing sparse event data requires compute-intensive convolutional recurrent units, complicating their integration into resource-constrained edge applications. Here, we propose the Sparse Event-based Efficient Detector (SEED) for efficient event-based object detection on neuromorphic processors. We introduce sparse convolutional recurrent learning, which achieves over 92% activation sparsity in recurrent processing, vastly reducing the cost for spatiotemporal reasoning on sparse event data. We validated our method on Prophesee's 1 Mpx and Gen1 event-based object detection datasets. Notably, SEED sets a new benchmark in computational efficiency for event-based object detection which requires long-term temporal learning. Compared to state-of-the-art methods, SEED significantly reduces synaptic operations while delivering higher or same-level mAP. Our hardware simulations showcase the critical role of SEED's hardware-aware design in achieving energy-efficient and low-latency neuromorphic processing.