V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving
This dataset addresses a gap for researchers in autonomous driving by enabling collaborative perception studies before real-world data is available, though it is incremental as it provides a new dataset rather than a novel method.
The authors tackled the lack of a public dataset for collaborative perception in autonomous driving by introducing V2X-Sim, a simulated multi-agent dataset with sensor recordings from vehicles and road-side units, which they benchmarked on detection, tracking, and segmentation tasks.
Vehicle-to-everything (V2X) communication techniques enable the collaboration between vehicles and many other entities in the neighboring environment, which could fundamentally improve the perception system for autonomous driving. However, the lack of a public dataset significantly restricts the research progress of collaborative perception. To fill this gap, we present V2X-Sim, a comprehensive simulated multi-agent perception dataset for V2X-aided autonomous driving. V2X-Sim provides: (1) \hl{multi-agent} sensor recordings from the road-side unit (RSU) and multiple vehicles that enable collaborative perception, (2) multi-modality sensor streams that facilitate multi-modality perception, and (3) diverse ground truths that support various perception tasks. Meanwhile, we build an open-source testbed and provide a benchmark for the state-of-the-art collaborative perception algorithms on three tasks, including detection, tracking and segmentation. V2X-Sim seeks to stimulate collaborative perception research for autonomous driving before realistic datasets become widely available. Our dataset and code are available at \url{https://ai4ce.github.io/V2X-Sim/}.