Objective-aware Traffic Simulation via Inverse Reinforcement Learning
This work addresses the challenge of non-stationary traffic dynamics for transportation planning and operation, offering a novel approach that improves simulation accuracy over conventional physical models.
The paper tackles the problem of accurately simulating vehicle behaviors in traffic by proposing an inverse reinforcement learning model that learns from real-world trajectories and recovers invariant reward functions, achieving superior performance and robustness compared to state-of-the-art methods in experiments on synthetic and real-world datasets.
Traffic simulators act as an essential component in the operating and planning of transportation systems. Conventional traffic simulators usually employ a calibrated physical car-following model to describe vehicles' behaviors and their interactions with traffic environment. However, there is no universal physical model that can accurately predict the pattern of vehicle's behaviors in different situations. A fixed physical model tends to be less effective in a complicated environment given the non-stationary nature of traffic dynamics. In this paper, we formulate traffic simulation as an inverse reinforcement learning problem, and propose a parameter sharing adversarial inverse reinforcement learning model for dynamics-robust simulation learning. Our proposed model is able to imitate a vehicle's trajectories in the real world while simultaneously recovering the reward function that reveals the vehicle's true objective which is invariant to different dynamics. Extensive experiments on synthetic and real-world datasets show the superior performance of our approach compared to state-of-the-art methods and its robustness to variant dynamics of traffic.