CtrlRAG: Black-box Adversarial Attacks Based on Masked Language Models in Retrieval-Augmented Language Generation
This addresses a critical security problem for RAG systems and LLM applications, exposing vulnerabilities in existing defenses and highlighting the need for more robust solutions, though it is incremental as it builds on known attack vectors.
The paper tackles the security threat of adversarial attacks on Retrieval-Augmented Generation (RAG) systems by proposing CtrlRAG, a black-box attack method that uses Masked Language Models to dynamically optimize malicious content, achieving superior performance over baseline methods in Emotional Manipulation and Hallucination Amplification objectives.
Retrieval-Augmented Generation (RAG) systems enhance Large Language Models (LLMs) by integrating external knowledge bases. However, this integration introduces a new security threat: adversaries can exploit the retrieval mechanism to inject malicious content into the knowledge base, thereby influencing the generated responses. Based on this attack vector, we propose CtrlRAG, a novel attack method designed for RAG system in the black-box setting, which aligns with real-world scenarios. Unlike existing attack methods, CtrlRAG introduces a perturbation mechanism using Masked Language Model (MLM) to dynamically optimize malicious content in response to changes in the retrieved context. Experimental results demonstrate that CtrlRAG outperforms three baseline methods in both Emotional Manipulation and Hallucination Amplification objectives. Furthermore, we evaluate three existing defense mechanisms, revealing their limited effectiveness against CtrlRAG and underscoring the urgent need for more robust defenses.