Hearing the Unspoken: Language Model Priors for Acoustic Adversarial Attacks
This work reveals a new vulnerability in real-time ASR systems by exploiting LLM-based predictive augmentation, posing a security threat to widely used speech interfaces.
The paper introduces the Semantic Gambit attack, which uses a large language model to provide predictive context to acoustic adversarial attacks on real-time ASR systems, achieving a 35.6% corpus-level Word Error Rate—a three-fold increase over the prior state-of-the-art.
Automatic Speech Recognition (ASR) systems operating in real-time settings must process acoustic input under strict temporal constraints, where transcription decisions are inherently made on incomplete information. This causal constraint serves as an information bottleneck on attackers, significantly limiting attack performance. Our new Semantic Gambit attack breaks this causal limitation by augmenting the adversary with predictive context derived from a Large Language Model in real-time. Our experiments show that this form of augmentation can elevate the corpus-level Word Error Rate to 35.6% -- a three-fold increase over the current state-of-the-art. Ultimately, this work reveals how common, low-latency LLM tooling can be exploited to systematically subvert real-time ASR pipelines.