Time-Ordered Recent Event (TORE) Volumes for Event Cameras
This addresses the challenge of efficiently processing event camera data for high-speed imaging applications, representing a novel method rather than an incremental improvement.
The paper tackles the problem of handling sparse data from event cameras by introducing Time-Ordered Recent Event (TORE) volumes, which compactly store raw spike timing information and dramatically improve state-of-the-art performance across tasks like denoising, reconstruction, classification, and human pose estimation.
Event cameras are an exciting, new sensor modality enabling high-speed imaging with extremely low-latency and wide dynamic range. Unfortunately, most machine learning architectures are not designed to directly handle sparse data, like that generated from event cameras. Many state-of-the-art algorithms for event cameras rely on interpolated event representations - obscuring crucial timing information, increasing the data volume, and limiting overall network performance. This paper details an event representation called Time-Ordered Recent Event (TORE) volumes. TORE volumes are designed to compactly store raw spike timing information with minimal information loss. This bio-inspired design is memory efficient, computationally fast, avoids time-blocking (i.e. fixed and predefined frame rates), and contains "local memory" from past data. The design is evaluated on a wide range of challenging tasks (e.g. event denoising, image reconstruction, classification, and human pose estimation) and is shown to dramatically improve state-of-the-art performance. TORE volumes are an easy-to-implement replacement for any algorithm currently utilizing event representations.