Safeguarding Privacy of Retrieval Data against Membership Inference Attacks: Is This Query Too Close to Home?
This addresses privacy risks in RAG systems for users handling sensitive data, though it is an incremental improvement focused on a specific attack vector.
The paper tackles the vulnerability of retrieval-augmented generation (RAG) systems to membership inference attacks (MIAs) by introducing a similarity-based detection framework, showing that a detect-and-hide strategy can successfully obfuscate attackers while maintaining data utility and system-agnostic adaptability.
Retrieval-augmented generation (RAG) mitigates the hallucination problem in large language models (LLMs) and has proven effective for personalized usages. However, delivering private retrieved documents directly to LLMs introduces vulnerability to membership inference attacks (MIAs), which try to determine whether the target data point exists in the private external database or not. Based on the insight that MIA queries typically exhibit high similarity to only one target document, we introduce a novel similarity-based MIA detection framework designed for the RAG system. With the proposed method, we show that a simple detect-and-hide strategy can successfully obfuscate attackers, maintain data utility, and remain system-agnostic against MIA. We experimentally prove its detection and defense against various state-of-the-art MIA methods and its adaptability to existing RAG systems.