NERVE: A Neuromorphic Vision and Radar Ensemble for Multi-Sensor Fusion Research
This dataset enables multi-modal fusion research for autonomous systems, but is an incremental contribution as a new benchmark dataset.
NERVE provides a 257-minute multi-sensor dataset (DVS, RGB-D, Radar) with ~9.6M annotations for 16 categories. On a DVS+Radar human detection task, recurrent models achieve 47.5% mAP and <1.8m distance error.
We present NERVE (Neuromorphic Vision and Radar Ensemble), a multi-sensor dataset comprising 257 minutes of synchronized recordings from five sensors: two Dynamic Vision Sensors (DVS), an RGB-D camera, and two Radar units (24GHz and 77GHz). Captured across 12 measurement days in office environments, NERVE contains around 600GB of uncompressed temporally aligned data with around 914,000 frames and around 9.6 million RGB COCO-formatted annotations covering 16 relevant object categories. To evaluate multi-modal fusion, we construct a DVS+Radar subset for human detection and distance estimation. Baseline experiments using feed-forward and recurrent detectors show that combining DVS with 77GHz Radar consistently improves detection, with recurrent models achieving up to 47.5% mAP and mean absolute Radar distance errors below 1.8m against LiDAR ground truth.