V2X-ReaLO: An Open Online Framework and Dataset for Cooperative Perception in Reality
This work addresses the problem of validating cooperative perception systems in dynamic real-world scenarios for autonomous vehicles, though it is incremental as it builds on existing datasets and methods.
The paper tackles the lack of real-world evaluation for V2X cooperative perception by introducing V2X-ReaLO, an open online framework deployed on real vehicles and infrastructure, which demonstrates the feasibility of intermediate fusion and provides a dataset with 25,028 test frames and 6,850 annotated key frames.
Cooperative perception enabled by Vehicle-to-Everything (V2X) communication holds significant promise for enhancing the perception capabilities of autonomous vehicles, allowing them to overcome occlusions and extend their field of view. However, existing research predominantly relies on simulated environments or static datasets, leaving the feasibility and effectiveness of V2X cooperative perception especially for intermediate fusion in real-world scenarios largely unexplored. In this work, we introduce V2X-ReaLO, an open online cooperative perception framework deployed on real vehicles and smart infrastructure that integrates early, late, and intermediate fusion methods within a unified pipeline and provides the first practical demonstration of online intermediate fusion's feasibility and performance under genuine real-world conditions. Additionally, we present an open benchmark dataset specifically designed to assess the performance of online cooperative perception systems. This new dataset extends V2X-Real dataset to dynamic, synchronized ROS bags and provides 25,028 test frames with 6,850 annotated key frames in challenging urban scenarios. By enabling real-time assessments of perception accuracy and communication lantency under dynamic conditions, V2X-ReaLO sets a new benchmark for advancing and optimizing cooperative perception systems in real-world applications. The codes and datasets will be released to further advance the field.