SPBA: Utilizing Speech Large Language Model for Backdoor Attacks on Speech Classification Models
This work addresses security vulnerabilities in speech-based human-computer interaction systems, presenting an incremental improvement in backdoor attack methods for speech classification models.
The paper tackles the problem of limited backdoor triggers in speech classification models by proposing SPBA, which uses a Speech Large Language Model to generate diverse triggers based on speech elements like timbre and emotion, and demonstrates significant effectiveness with exceptional attack metrics in experiments on keyword spotting and speaker verification tasks.
Deep speech classification tasks, including keyword spotting and speaker verification, are vital in speech-based human-computer interaction. Recently, the security of these technologies has been revealed to be susceptible to backdoor attacks. Specifically, attackers use noisy disruption triggers and speech element triggers to produce poisoned speech samples that train models to become vulnerable. However, these methods typically create only a limited number of backdoors due to the inherent constraints of the trigger function. In this paper, we propose that speech backdoor attacks can strategically focus on speech elements such as timbre and emotion, leveraging the Speech Large Language Model (SLLM) to generate diverse triggers. Increasing the number of triggers may disproportionately elevate the poisoning rate, resulting in higher attack costs and a lower success rate per trigger. We introduce the Multiple Gradient Descent Algorithm (MGDA) as a mitigation strategy to address this challenge. The proposed attack is called the Speech Prompt Backdoor Attack (SPBA). Building on this foundation, we conducted attack experiments on two speech classification tasks, demonstrating that SPBA shows significant trigger effectiveness and achieves exceptional performance in attack metrics.