Multi-V2X: A Large Scale Multi-modal Multi-penetration-rate Dataset for Cooperative Perception
This dataset addresses a critical gap for researchers and developers in autonomous driving by enabling more realistic testing of cooperative perception algorithms, though it is incremental as it builds on existing simulation methods.
The authors tackled the lack of diverse and realistic datasets for cooperative perception in V2X systems by introducing Multi-V2X, a large-scale dataset with 549k RGB frames, 146k LiDAR frames, and 4.2 million annotated 3D boxes, featuring variable CAV penetration rates up to 86.21% and up to 31 agents.
Cooperative perception through vehicle-to-everything (V2X) has garnered significant attention in recent years due to its potential to overcome occlusions and enhance long-distance perception. Great achievements have been made in both datasets and algorithms. However, existing real-world datasets are limited by the presence of few communicable agents, while synthetic datasets typically cover only vehicles. More importantly, the penetration rate of connected and autonomous vehicles (CAVs) , a critical factor for the deployment of cooperative perception technologies, has not been adequately addressed. To tackle these issues, we introduce Multi-V2X, a large-scale, multi-modal, multi-penetration-rate dataset for V2X perception. By co-simulating SUMO and CARLA, we equip a substantial number of cars and roadside units (RSUs) in simulated towns with sensor suites, and collect comprehensive sensing data. Datasets with specified CAV penetration rates can be obtained by masking some equipped cars as normal vehicles. In total, our Multi-V2X dataset comprises 549k RGB frames, 146k LiDAR frames, and 4,219k annotated 3D bounding boxes across six categories. The highest possible CAV penetration rate reaches 86.21%, with up to 31 agents in communication range, posing new challenges in selecting agents to collaborate with. We provide comprehensive benchmarks for cooperative 3D object detection tasks. Our data and code are available at https://github.com/RadetzkyLi/Multi-V2X .