Bot Wars Evolved: Orchestrating Competing LLMs in a Counterstrike Against Phone Scams
This addresses phone scams for potential victims and law enforcement, but it is incremental as it builds on existing LLM capabilities for a specific domain.
The paper tackles phone scams by developing 'Bot Wars,' a framework that uses LLMs as scam-baiters in simulated adversarial dialogues, achieving effectiveness validated against 3,200 scam dialogues and 179 hours of human interactions.
We present "Bot Wars," a framework using Large Language Models (LLMs) scam-baiters to counter phone scams through simulated adversarial dialogues. Our key contribution is a formal foundation for strategy emergence through chain-of-thought reasoning without explicit optimization. Through a novel two-layer prompt architecture, our framework enables LLMs to craft demographically authentic victim personas while maintaining strategic coherence. We evaluate our approach using a dataset of 3,200 scam dialogues validated against 179 hours of human scam-baiting interactions, demonstrating its effectiveness in capturing complex adversarial dynamics. Our systematic evaluation through cognitive, quantitative, and content-specific metrics shows that GPT-4 excels in dialogue naturalness and persona authenticity, while Deepseek demonstrates superior engagement sustainability.