Event Transformer. A sparse-aware solution for efficient event data processing
This work addresses the need for efficient and accurate event data processing for applications in low-resource and challenging environments, representing a novel method for a known bottleneck.
The paper tackles the problem of inefficient and inaccurate processing of event camera data by proposing Event Transformer (EvT), a framework that achieves better or comparable accuracy to state-of-the-art methods while requiring significantly less computational resources, enabling minimal latency on both GPU and CPU.
Event cameras are sensors of great interest for many applications that run in low-resource and challenging environments. They log sparse illumination changes with high temporal resolution and high dynamic range, while they present minimal power consumption. However, top-performing methods often ignore specific event-data properties, leading to the development of generic but computationally expensive algorithms. Efforts toward efficient solutions usually do not achieve top-accuracy results for complex tasks. This work proposes a novel framework, Event Transformer (EvT), that effectively takes advantage of event-data properties to be highly efficient and accurate. We introduce a new patch-based event representation and a compact transformer-like architecture to process it. EvT is evaluated on different event-based benchmarks for action and gesture recognition. Evaluation results show better or comparable accuracy to the state-of-the-art while requiring significantly less computation resources, which makes EvT able to work with minimal latency both on GPU and CPU.