ReactSim-Bench: Benchmarking Reactive Behavior World Model Simulation in Autonomous Driving
For researchers developing data-driven simulators for autonomous driving, this benchmark addresses the lack of direct evaluation of reactive behavior, which is critical for safe AV development.
The paper introduces ReactSim-Bench, a benchmark for evaluating reactive capability of behavior world model simulators in autonomous driving. It decouples AV and agent control, uses diverse AV behaviors, and evaluates state-of-the-art models across multiple architectures, finding that reactive capability varies significantly and replan frequency impacts performance.
Reactive capability is a key property of data-driven behavior world model simulators for autonomous driving simulation systems. With this capability, simulated world agents can respond feasibly to autonomous vehicle (AV) behaviors that differ from the log. However, existing behavior simulation benchmarks do not directly measure reactive capability. They often let the simulator jointly control the AV and surrounding agents and evaluate realism through log similarity or open-loop prediction metrics. In this work, we introduce ReactSim-Bench for evaluating the reactive capability of behavior world model simulation in autonomous driving. We decouple the control of agents and the AV, using AV behaviors that differ from the log and require agents to respond as independent AV inputs. To obtain these AV behaviors, we construct a pipeline that uses an AV planner model to generate candidate behaviors and filters the data using rules and manual verification. Collision metrics, map-based metrics, and kinematic feasibility metrics are used to evaluate the safety and rule compliance of reactive responses. We construct 2,636 test scenarios with three categories and conduct a systematic evaluation of state-of-the-art models across multiple architectures, including Transformer-based, diffusion-based, and next-token-prediction-based models. We further analyze how replan frequency affects performance and provide insights for future studies.