Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks
This work addresses the challenge of efficient optical flow estimation for event-based cameras, which is crucial for applications like high-speed motion detection and low-light navigation, representing an incremental improvement over existing methods.
The authors tackled the problem of estimating optical flow from event-based camera outputs, which are asynchronous and sparse, by proposing Spike-FlowNet, a hybrid neural network combining SNNs and ANNs. The result is that Spike-FlowNet outperforms ANN-based methods in optical flow prediction while offering significant computational efficiency, as demonstrated on the MVSEC dataset.
Event-based cameras display great potential for a variety of tasks such as high-speed motion detection and navigation in low-light environments where conventional frame-based cameras suffer critically. This is attributed to their high temporal resolution, high dynamic range, and low-power consumption. However, conventional computer vision methods as well as deep Analog Neural Networks (ANNs) are not suited to work well with the asynchronous and discrete nature of event camera outputs. Spiking Neural Networks (SNNs) serve as ideal paradigms to handle event camera outputs, but deep SNNs suffer in terms of performance due to the spike vanishing phenomenon. To overcome these issues, we present Spike-FlowNet, a deep hybrid neural network architecture integrating SNNs and ANNs for efficiently estimating optical flow from sparse event camera outputs without sacrificing the performance. The network is end-to-end trained with self-supervised learning on Multi-Vehicle Stereo Event Camera (MVSEC) dataset. Spike-FlowNet outperforms its corresponding ANN-based method in terms of the optical flow prediction capability while providing significant computational efficiency.