EventNet: Asynchronous Recursive Event Processing
This work addresses the challenge of efficiently using event camera data for real-time vision applications, representing an incremental improvement in adapting neural networks to sparse inputs.
The authors tackled the problem of processing sparse, asynchronous event streams from event cameras with existing dense neural networks by proposing EventNet, a recursive neural network that processes events individually, achieving real-time performance of over 1 million events per second on a standard CPU.
Event cameras are bio-inspired vision sensors that mimic retinas to asynchronously report per-pixel intensity changes rather than outputting an actual intensity image at regular intervals. This new paradigm of image sensor offers significant potential advantages; namely, sparse and non-redundant data representation. Unfortunately, however, most of the existing artificial neural network architectures, such as a CNN, require dense synchronous input data, and therefore, cannot make use of the sparseness of the data. We propose EventNet, a neural network designed for real-time processing of asynchronous event streams in a recursive and event-wise manner. EventNet models dependence of the output on tens of thousands of causal events recursively using a novel temporal coding scheme. As a result, at inference time, our network operates in an event-wise manner that is realized with very few sum-of-the-product operations---look-up table and temporal feature aggregation---which enables processing of 1 mega or more events per second on standard CPU. In experiments using real data, we demonstrated the real-time performance and robustness of our framework.