Dataset and Benchmark: Novel Sensors for Autonomous Vehicle Perception
This dataset facilitates research on novel sensors for autonomous vehicles, but it is incremental as it builds on existing sensor types by combining them in a new dataset.
The paper introduces the NSAVP dataset, which includes stereo event, thermal, monochrome, and RGB cameras to address challenges like low-light and adverse weather in autonomous vehicle perception, and provides benchmarking for place recognition to demonstrate sensor capabilities.
Conventional cameras employed in autonomous vehicle (AV) systems support many perception tasks, but are challenged by low-light or high dynamic range scenes, adverse weather, and fast motion. Novel sensors, such as event and thermal cameras, offer capabilities with the potential to address these scenarios, but they remain to be fully exploited. This paper introduces the Novel Sensors for Autonomous Vehicle Perception (NSAVP) dataset to facilitate future research on this topic. The dataset was captured with a platform including stereo event, thermal, monochrome, and RGB cameras as well as a high precision navigation system providing ground truth poses. The data was collected by repeatedly driving two ~8 km routes and includes varied lighting conditions and opposing viewpoint perspectives. We provide benchmarking experiments on the task of place recognition to demonstrate challenges and opportunities for novel sensors to enhance critical AV perception tasks. To our knowledge, the NSAVP dataset is the first to include stereo thermal cameras together with stereo event and monochrome cameras. The dataset and supporting software suite is available at: https://umautobots.github.io/nsavp