A Differentiable Recurrent Surface for Asynchronous Event-Based Data
This addresses the challenge of processing sparse event-based vision data for computer vision tasks, offering a novel method that outperforms hand-crafted approaches.
The paper tackles the problem of integrating asynchronous event data from Dynamic Vision Sensors into usable frames by proposing Matrix-LSTM, a grid of LSTM cells that learns task-dependent event-surfaces end-to-end. It improves state-of-the-art in event-based object classification on the N-Cars dataset and shows good flexibility in optical flow estimation on the MVSEC benchmark.
Dynamic Vision Sensors (DVSs) asynchronously stream events in correspondence of pixels subject to brightness changes. Differently from classic vision devices, they produce a sparse representation of the scene. Therefore, to apply standard computer vision algorithms, events need to be integrated into a frame or event-surface. This is usually attained through hand-crafted grids that reconstruct the frame using ad-hoc heuristics. In this paper, we propose Matrix-LSTM, a grid of Long Short-Term Memory (LSTM) cells that efficiently process events and learn end-to-end task-dependent event-surfaces. Compared to existing reconstruction approaches, our learned event-surface shows good flexibility and expressiveness on optical flow estimation on the MVSEC benchmark and it improves the state-of-the-art of event-based object classification on the N-Cars dataset.