AdvDO: Realistic Adversarial Attacks for Trajectory Prediction
This addresses a critical safety problem for autonomous vehicles by exposing vulnerabilities in data-driven prediction systems, with incremental contributions in benchmarking and mitigation.
The paper tackles the lack of adversarial robustness studies in trajectory prediction for autonomous vehicles by proposing an optimization-based attack framework that generates realistic adversarial trajectories, increasing prediction errors by over 50% and 37% on key metrics and causing unsafe driving behaviors in simulation.
Trajectory prediction is essential for autonomous vehicles (AVs) to plan correct and safe driving behaviors. While many prior works aim to achieve higher prediction accuracy, few study the adversarial robustness of their methods. To bridge this gap, we propose to study the adversarial robustness of data-driven trajectory prediction systems. We devise an optimization-based adversarial attack framework that leverages a carefully-designed differentiable dynamic model to generate realistic adversarial trajectories. Empirically, we benchmark the adversarial robustness of state-of-the-art prediction models and show that our attack increases the prediction error for both general metrics and planning-aware metrics by more than 50% and 37%. We also show that our attack can lead an AV to drive off road or collide into other vehicles in simulation. Finally, we demonstrate how to mitigate the adversarial attacks using an adversarial training scheme.