Scaling Datasets for Multi-Sensor, Multi-Agent, and Multi-Domain Learning in Autonomous Systems
This work addresses the lack of large-scale, diverse datasets for training and evaluating multi-agent autonomous systems, but the contribution is incremental as it builds on existing simulators and frameworks.
The authors present a modular pipeline for generating terabyte-scale, labeled multi-sensor, multi-agent, and multi-domain datasets for autonomous systems, demonstrating its utility through perception and fusion studies.
Existing datasets cannot support large-scale learning in multi-agent, multi-sensor, or multi-domain autonomy, where diversity and coordination are essential. We present a modular dataset generation pipeline that creates terabyte-scale, ground-truth-labeled data for ground, aerial, and infrastructure-based systems using the AVstack framework and CARLA simulator. Supporting single- and multi-agent configurations with flexible sensor suites, the pipeline enables controllable experimentation across challenging conditions. Representative perception and fusion studies show how generated data can support application-specific training and collaborative autonomy.