Who Stole Your Data? A Method for Detecting Unauthorized RAG Theft
This addresses intellectual property protection for AI systems using RAG, offering a foundational but incremental framework.
The paper tackles the problem of unauthorized data theft in retrieval-augmented generation (RAG) systems by introducing a dual-layered watermarking system and a dataset for detection, achieving effectiveness across various conditions with resilience to evasion.
Retrieval-augmented generation (RAG) enhances Large Language Models (LLMs) by mitigating hallucinations and outdated information issues, yet simultaneously facilitates unauthorized data appropriation at scale. This paper addresses this challenge through two key contributions. First, we introduce RPD, a novel dataset specifically designed for RAG plagiarism detection that encompasses diverse professional domains and writing styles, overcoming limitations in existing resources. Second, we develop a dual-layered watermarking system that embeds protection at both semantic and lexical levels, complemented by an interrogator-detective framework that employs statistical hypothesis testing on accumulated evidence. Extensive experimentation demonstrates our approach's effectiveness across varying query volumes, defense prompts, and retrieval parameters, while maintaining resilience against adversarial evasion techniques. This work establishes a foundational framework for intellectual property protection in retrieval-augmented AI systems.