eCARLA-scenes: A synthetically generated dataset for event-based optical flow prediction
This work addresses a data scarcity problem for researchers in robotics and autonomous systems, though it is incremental as it builds on existing simulation tools.
The authors tackled the lack of diverse and scalable event-based datasets for optical flow prediction by introducing eCARLA-scenes, a synthetically generated dataset using the CARLA simulator, which includes tools for data processing and aims to support applications in autonomous vehicle navigation with Spiking Neural Networks.
The joint use of event-based vision and Spiking Neural Networks (SNNs) is expected to have a large impact in robotics in the near future, in tasks such as, visual odometry and obstacle avoidance. While researchers have used real-world event datasets for optical flow prediction (mostly captured with Unmanned Aerial Vehicles (UAVs)), these datasets are limited in diversity, scalability, and are challenging to collect. Thus, synthetic datasets offer a scalable alternative by bridging the gap between reality and simulation. In this work, we address the lack of datasets by introducing eWiz, a comprehensive library for processing event-based data. It includes tools for data loading, augmentation, visualization, encoding, and generation of training data, along with loss functions and performance metrics. We further present a synthetic event-based datasets and data generation pipelines for optical flow prediction tasks. Built on top of eWiz, eCARLA-scenes makes use of the CARLA simulator to simulate self-driving car scenarios. The ultimate goal of this dataset is the depiction of diverse environments while laying a foundation for advancing event-based camera applications in autonomous field vehicle navigation, paving the way for using SNNs on neuromorphic hardware such as the Intel Loihi.