ScamAgents: How AI Agents Can Simulate Human-Level Scam Calls
This work addresses a critical security issue for AI developers and policymakers by exposing vulnerabilities in conversational AI safety, though it is incremental in building on existing LLM capabilities.
The paper tackled the problem of AI misuse by developing ScamAgent, an autonomous agent that generates realistic scam call scripts, and found that current LLM safety measures are ineffective against such multi-turn threats, with the agent bypassing safeguards to create fully automated scam pipelines.
Large Language Models (LLMs) have demonstrated impressive fluency and reasoning capabilities, but their potential for misuse has raised growing concern. In this paper, we present ScamAgent, an autonomous multi-turn agent built on top of LLMs, capable of generating highly realistic scam call scripts that simulate real-world fraud scenarios. Unlike prior work focused on single-shot prompt misuse, ScamAgent maintains dialogue memory, adapts dynamically to simulated user responses, and employs deceptive persuasion strategies across conversational turns. We show that current LLM safety guardrails, including refusal mechanisms and content filters, are ineffective against such agent-based threats. Even models with strong prompt-level safeguards can be bypassed when prompts are decomposed, disguised, or delivered incrementally within an agent framework. We further demonstrate the transformation of scam scripts into lifelike voice calls using modern text-to-speech systems, completing a fully automated scam pipeline. Our findings highlight an urgent need for multi-turn safety auditing, agent-level control frameworks, and new methods to detect and disrupt conversational deception powered by generative AI.