WHALES: A Multi-Agent Scheduling Dataset for Enhanced Cooperation in Autonomous Driving
This addresses the problem of limited real-world V2X interaction data for researchers in autonomous driving, though it is incremental as it builds on existing cooperative perception work.
The authors tackled the lack of datasets for cooperative perception in autonomous driving by introducing WHALES, a large-scale V2X dataset with 2.01 million annotated 3D objects and an average of 8.4 agents per scene, and proposed a scheduling baseline that improves perception performance over SOTA methods.
Cooperative perception research is hindered by the limited availability of datasets that capture the complexity of real-world Vehicle-to-Everything (V2X) interactions, particularly under dynamic communication constraints. To address this gap, we introduce WHALES (Wireless enhanced Autonomous vehicles with Large number of Engaged agents), the first large-scale V2X dataset explicitly designed to benchmark communication-aware agent scheduling and scalable cooperative perception. WHALES introduces a new benchmark that enables state-of-the-art (SOTA) research in communication-aware cooperative perception, featuring an average of 8.4 cooperative agents per scene and 2.01 million annotated 3D objects across diverse traffic scenarios. It incorporates detailed communication metadata to emulate real-world communication bottlenecks, enabling rigorous evaluation of scheduling strategies. To further advance the field, we propose the Coverage-Aware Historical Scheduler (CAHS), a novel scheduling baseline that selects agents based on historical viewpoint coverage, improving perception performance over existing SOTA methods. WHALES bridges the gap between simulated and real-world V2X challenges, providing a robust framework for exploring perception-scheduling co-design, cross-data generalization, and scalability limits. The WHALES dataset and code are available at https://github.com/chensiweiTHU/WHALES.