Unsupervised Anomaly Detection in Multi-Agent Trajectory Prediction via Transformer-Based Models
This addresses the challenge of detecting rare and complex safety risks in autonomous driving without labeled data, offering actionable insights for simulation and testing, though it is incremental in improving anomaly detection methods.
The paper tackles the problem of identifying safety-critical anomalies in multi-agent trajectory prediction for autonomous driving by proposing an unsupervised Transformer-based framework that measures deviations through prediction residuals, achieving high physical alignment and identifying 388 unique anomalies missed by baselines.
Identifying safety-critical scenarios is essential for autonomous driving, but the rarity of such events makes supervised labeling impractical. Traditional rule-based metrics like Time-to-Collision are too simplistic to capture complex interaction risks, and existing methods lack a systematic way to verify whether statistical anomalies truly reflect physical danger. To address this gap, we propose an unsupervised anomaly detection framework based on a multi-agent Transformer that models normal driving and measures deviations through prediction residuals. A dual evaluation scheme has been proposed to assess both detection stability and physical alignment: Stability is measured using standard ranking metrics in which Kendall Rank Correlation Coefficient captures rank agreement and Jaccard index captures the consistency of the top-K selected items; Physical alignment is assessed through correlations with established Surrogate Safety Measures (SSM). Experiments on the NGSIM dataset demonstrate our framework's effectiveness: We show that the maximum residual aggregator achieves the highest physical alignment while maintaining stability. Furthermore, our framework identifies 388 unique anomalies missed by Time-to-Collision and statistical baselines, capturing subtle multi-agent risks like reactive braking under lateral drift. The detected anomalies are further clustered into four interpretable risk types, offering actionable insights for simulation and testing.