RAG-Pull: Imperceptible Attacks on RAG Systems for Code Generation
This exposes a critical security flaw in RAG systems for code generation, posing risks to developers and users reliant on these models for safe code output.
The paper tackles the vulnerability of Retrieval-Augmented Generation (RAG) systems in code generation by introducing RAG-Pull, a black-box attack that uses hidden UTF characters in queries or repositories to redirect retrieval toward malicious code, achieving near-perfect success in breaking safety alignment and introducing exploitable vulnerabilities like remote code execution.
Retrieval-Augmented Generation (RAG) increases the reliability and trustworthiness of the LLM response and reduces hallucination by eliminating the need for model retraining. It does so by adding external data into the LLM's context. We develop a new class of black-box attack, RAG-Pull, that inserts hidden UTF characters into queries or external code repositories, redirecting retrieval toward malicious code, thereby breaking the models' safety alignment. We observe that query and code perturbations alone can shift retrieval toward attacker-controlled snippets, while combined query-and-target perturbations achieve near-perfect success. Once retrieved, these snippets introduce exploitable vulnerabilities such as remote code execution and SQL injection. RAG-Pull's minimal perturbations can alter the model's safety alignment and increase preference towards unsafe code, therefore opening up a new class of attacks on LLMs.