AirGapAgent: Protecting Privacy-Conscious Conversational Agents
This addresses privacy risks for users of sensitive data-handling conversational agents, though it is an incremental improvement focused on a specific threat model.
The paper tackles the problem of adversarial context hijacking that tricks LLM-based conversational agents into leaking private data, and shows that their AirGapAgent method achieves 97% protection, rendering attacks ineffective compared to a baseline dropping to 45%.
The growing use of large language model (LLM)-based conversational agents to manage sensitive user data raises significant privacy concerns. While these agents excel at understanding and acting on context, this capability can be exploited by malicious actors. We introduce a novel threat model where adversarial third-party apps manipulate the context of interaction to trick LLM-based agents into revealing private information not relevant to the task at hand. Grounded in the framework of contextual integrity, we introduce AirGapAgent, a privacy-conscious agent designed to prevent unintended data leakage by restricting the agent's access to only the data necessary for a specific task. Extensive experiments using Gemini, GPT, and Mistral models as agents validate our approach's effectiveness in mitigating this form of context hijacking while maintaining core agent functionality. For example, we show that a single-query context hijacking attack on a Gemini Ultra agent reduces its ability to protect user data from 94% to 45%, while an AirGapAgent achieves 97% protection, rendering the same attack ineffective.