Attacker's Noise Can Manipulate Your Audio-based LLM in the Real World
This work highlights critical security vulnerabilities in audio-based LLMs, posing risks for users in real-world settings, though it is incremental in building on existing adversarial attack research.
The paper demonstrates that adversaries can craft stealthy audio perturbations to manipulate audio-based large language models (ALLMs) into targeted behaviors like wake-keyword responses or harmful actions, and that playing adversarial background noise can significantly degrade response quality, with attacks scalable to real-world scenarios affecting other users.
This paper investigates the real-world vulnerabilities of audio-based large language models (ALLMs), such as Qwen2-Audio. We first demonstrate that an adversary can craft stealthy audio perturbations to manipulate ALLMs into exhibiting specific targeted behaviors, such as eliciting responses to wake-keywords (e.g., "Hey Qwen"), or triggering harmful behaviors (e.g. "Change my calendar event"). Subsequently, we show that playing adversarial background noise during user interaction with the ALLMs can significantly degrade the response quality. Crucially, our research illustrates the scalability of these attacks to real-world scenarios, impacting other innocent users when these adversarial noises are played through the air. Further, we discuss the transferrability of the attack, and potential defensive measures.