Ithaca365: Dataset and Driving Perception under Repeated and Challenging Weather Conditions
This dataset addresses the need for robust autonomous driving perception in challenging conditions like snow and rain, though it is incremental as it builds on existing data collection methods.
The authors tackled the problem of robust perception for self-driving cars under varied weather conditions by introducing the Ithaca365 dataset, collected repeatedly along a 15 km route with diverse scenes, weather, and traffic, and demonstrated its utility through baseline analyses in tasks like amodal segmentation and 3D object detection.
Advances in perception for self-driving cars have accelerated in recent years due to the availability of large-scale datasets, typically collected at specific locations and under nice weather conditions. Yet, to achieve the high safety requirement, these perceptual systems must operate robustly under a wide variety of weather conditions including snow and rain. In this paper, we present a new dataset to enable robust autonomous driving via a novel data collection process - data is repeatedly recorded along a 15 km route under diverse scene (urban, highway, rural, campus), weather (snow, rain, sun), time (day/night), and traffic conditions (pedestrians, cyclists and cars). The dataset includes images and point clouds from cameras and LiDAR sensors, along with high-precision GPS/INS to establish correspondence across routes. The dataset includes road and object annotations using amodal masks to capture partial occlusions and 3D bounding boxes. We demonstrate the uniqueness of this dataset by analyzing the performance of baselines in amodal segmentation of road and objects, depth estimation, and 3D object detection. The repeated routes opens new research directions in object discovery, continual learning, and anomaly detection. Link to Ithaca365: https://ithaca365.mae.cornell.edu/