Evaluating Generative Vehicle Trajectory Models for Traffic Intersection Dynamics
This work addresses a gap for traffic engineers by providing better evaluation tools for generative models in traffic intersection dynamics, though it is incremental as it builds on existing models and datasets.
The paper tackled the problem of evaluating generative vehicle trajectory models for traffic intersections by developing a comprehensive analytics tool that assesses models in live microsimulation scenarios with traffic engineering-specific metrics, revealing that even models with low reconstruction errors can generate trajectories that violate traffic rules.
Traffic Intersections are vital to urban road networks as they regulate the movement of people and goods. However, they are regions of conflicting trajectories and are prone to accidents. Deep Generative models of traffic dynamics at signalized intersections can greatly help traffic authorities better understand the efficiency and safety aspects. At present, models are evaluated on computational metrics that primarily look at trajectory reconstruction errors. They are not evaluated online in a `live' microsimulation scenario. Further, these metrics do not adequately consider traffic engineering-specific concerns such as red-light violations, unallowed stoppage, etc. In this work, we provide a comprehensive analytics tool to train, run, and evaluate models with metrics that give better insights into model performance from a traffic engineering point of view. We train a state-of-the-art multi-vehicle trajectory forecasting model on a large dataset collected by running a calibrated scenario of a real-world urban intersection. We then evaluate the performance of the prediction models, online in a microsimulator, under unseen traffic conditions. We show that despite using ideally-behaved trajectories as input, and achieving low trajectory reconstruction errors, the generated trajectories show behaviors that break traffic rules. We introduce new metrics to evaluate such undesired behaviors and present our results.