V2X-Real: a Large-Scale Dataset for Vehicle-to-Everything Cooperative Perception
This dataset addresses a critical gap for researchers and developers in autonomous driving by providing a real-world resource for V2X cooperative perception, though it is incremental as it builds on existing datasets by extending to mixed vehicle-infrastructure cooperation.
The paper tackles the lack of real-world datasets for Vehicle-to-Everything (V2X) cooperative perception by introducing V2X-Real, a large-scale dataset with 33K LiDAR frames, 171K camera data, and over 1.2M annotated bounding boxes across 10 categories in challenging urban scenarios, enabling comprehensive benchmarks for SOTA methods.
Recent advancements in Vehicle-to-Everything (V2X) technologies have enabled autonomous vehicles to share sensing information to see through occlusions, greatly boosting the perception capability. However, there are no real-world datasets to facilitate the real V2X cooperative perception research -- existing datasets either only support Vehicle-to-Infrastructure cooperation or Vehicle-to-Vehicle cooperation. In this paper, we present V2X-Real, a large-scale dataset that includes a mixture of multiple vehicles and smart infrastructure to facilitate the V2X cooperative perception development with multi-modality sensing data. Our V2X-Real is collected using two connected automated vehicles and two smart infrastructure, which are all equipped with multi-modal sensors including LiDAR sensors and multi-view cameras. The whole dataset contains 33K LiDAR frames and 171K camera data with over 1.2M annotated bounding boxes of 10 categories in very challenging urban scenarios. According to the collaboration mode and ego perspective, we derive four types of datasets for Vehicle-Centric, Infrastructure-Centric, Vehicle-to-Vehicle, and Infrastructure-to-Infrastructure cooperative perception. Comprehensive multi-class multi-agent benchmarks of SOTA cooperative perception methods are provided. The V2X-Real dataset and codebase are available at https://mobility-lab.seas.ucla.edu/v2x-real.