Michael Kühr

Michael Kühr, Mevlüt Yildirim, Maximilian Luedecke et al.

Traffic sign classification is a crucial task for autonomous vehicles, and numerous attacks against it have been identified. A majority of physical adversarial attacks involve attaching patches to traffic signs or projecting perturbations on them. While they demonstrate high effectiveness, they are perceptible to humans. At the same time, light-based attacks outside the human visible spectrum are known but have limitations in their dynamic adaptability. We propose a persistence-of-vision-based attack that operates in the near-infrared light spectrum. With the possibility of showing dynamic, remotely triggered content, this allows a stealthy physical adversarial attack against traffic sign classification. By identifying the optimal position through digital simulation, we conduct extensive real-world evaluations using two different traffic signs, 12 machine learning models from different families, multiple distances up to 20 meters, and varying illumination conditions. Our evaluation shows high attack success rates across our test scenarios. We propose near-infrared cutoff filters and a software-based detection mechanism as defenses, and tackle limitations of the near-infrared persistence of vision display by prototyping a human-visible RGB version of it.

Pedram MohajerAnsari, Amir Salarpour, Michael Kühr et al.

Autonomous vehicles (AVs) rely on deep neural networks (DNNs) for critical tasks such as traffic sign recognition (TSR), automated lane centering (ALC), and vehicle detection (VD). However, these models are vulnerable to attacks that can cause misclassifications and compromise safety. Traditional defense mechanisms, including adversarial training, often degrade benign accuracy and fail to generalize against unseen attacks. In this work, we introduce Vehicle Vision Language Models (V2LMs), fine-tuned vision-language models specialized for AV perception. Our findings demonstrate that V2LMs inherently exhibit superior robustness against unseen attacks without requiring adversarial training, maintaining significantly higher accuracy than conventional DNNs under adversarial conditions. We evaluate two deployment strategies: Solo Mode, where individual V2LMs handle specific perception tasks, and Tandem Mode, where a single unified V2LM is fine-tuned for multiple tasks simultaneously. Experimental results reveal that DNNs suffer performance drops of 33% to 46% under attacks, whereas V2LMs maintain adversarial accuracy with reductions of less than 8% on average. The Tandem Mode further offers a memory-efficient alternative while achieving comparable robustness to Solo Mode. We also explore integrating V2LMs as parallel components to AV perception to enhance resilience against adversarial threats. Our results suggest that V2LMs offer a promising path toward more secure and resilient AV perception systems.