An Empirical Study of Privacy Leakage Chains via Prompt Injection in Black-Box Chatbot Environments
For security researchers and developers of LLM-based chatbot agents, this work highlights a practical privacy-leakage attack vector, though it is an incremental extension of known prompt injection and jailbreak techniques.
This paper studies privacy-leakage attack chains via indirect prompt injection in black-box LLM-based chatbot agents, introducing a new technique called exemplification. It demonstrates a feasible path combining prompt injection, jailbreak-style instruction steering, and web-tool invocation to exfiltrate data.
LLM-based chatbot agents increasingly process user requests by combining natural-language reasoning with external tools such as web browsing. These capabilities improve usability, but they also create attack surfaces when untrusted external content is processed as part of a user' s task. This paper studies a privacy-leakage attack chain based on indirect prompt injection in black-box chatbot environments, where the attacker has no access to model weights, system prompts, or agent implementation details including how a trajectory is actually managed during its processing for a query. We first analyze how an attacker can hijack an agent' s intended task by crafting external content that appears benign to the victim while inducing the agent to execute an attacker-defined objective. We then evaluate a new prompt-injection technique, called exemplification, which uses a bridge in the external content to reframe the user prompt and the benign beginning of the retrieved page as few-shot examples before appending the attacker' s objective. We compare its attack success rate with a prior fake-completion technique. Finally, we demonstrate a proof-of-concept data-exfiltration chain using fictitious personal information in a controlled setting. Our results suggest that prompt injection, jailbreak-style instruction steering, and web-tool invocation can be combined into a feasible privacy-leakage path in deployed chatbot agents.