Event Quality Score (EQS): Assessing the Realism of Simulated Event Camera Streams via Distances in Latent Space
This work addresses the problem of improving event camera simulators for researchers and developers in computer vision, but it is incremental as it builds on existing deep feature methods for image comparison.
The paper tackles the challenge of assessing the realism of simulated event camera data, which is crucial due to the scarcity of labeled real datasets, and introduces the Event Quality Score (EQS) metric that shows higher scores correlate with improved generalization to real-world data in sim-to-real experiments on the DSEC driving dataset.
Event cameras promise a paradigm shift in vision sensing with their low latency, high dynamic range, and asynchronous nature of events. Unfortunately, the scarcity of high-quality labeled datasets hinders their widespread adoption in deep learning-driven computer vision. To mitigate this, several simulators have been proposed to generate synthetic event data for training models for detection and estimation tasks. However, the fundamentally different sensor design of event cameras compared to traditional frame-based cameras poses a challenge for accurate simulation. As a result, most simulated data fail to mimic data captured by real event cameras. Inspired by existing work on using deep features for image comparison, we introduce event quality score (EQS), a quality metric that utilizes activations of the RVT architecture. Through sim-to-real experiments on the DSEC driving dataset, it is shown that a higher EQS implies improved generalization to real-world data after training on simulated events. Thus, optimizing for EQS can lead to developing more realistic event camera simulators, effectively reducing the simulation gap. EQS is available at https://github.com/eventbasedvision/EQS.