Memory as an Attack Surface in LLM Agents: A Study on Multiple-Choice Question Answering
It identifies a new vulnerability in LLM agents with memory, relevant for security and trustworthiness of AI systems.
The paper shows that memory manipulation in LLM-based agents for multiple-choice question answering can cause the agent to select incorrect options even when the current query is clean, with measurable changes in accuracy and attack success rate.
AI agents extend conventional large language model (LLM) applications by integrating language understanding with task execution, external tool use, and memory mechanisms. While memory allows agents to retain prior interactions and provide more personalized and context-aware responses, it also introduces a new vulnerability: information stored in memory can influence future outputs even when the current query is clean. In this paper, we investigate memory manipulation in LLM-based agents for multiple-choice question answering. We first design and implement an LLM-based AI agent with an external memory component that stores and retrieves task-relevant information. We then introduce basic memory manipulation scenarios in which misleading or corrupted memories are inserted into the agent before it answers multiple-choice questions. Using a controlled experimental setup, we compare the agent's performance before and after memory manipulation and measure changes in answer accuracy, attack success rate, and selection of manipulated options. Our results show that even simple memory manipulations can noticeably affect the agent's final answers, causing it to select incorrect options despite receiving clean and well-formed questions.