When AI Agents Collude Online: Financial Fraud Risks by Collaborative LLM Agents on Social Platforms
This work addresses the problem of AI-driven financial fraud on social platforms, which poses risks to online security and trust, though it is incremental in building on existing multi-agent and fraud detection research.
The study investigated the risks of collective financial fraud in multi-agent systems using LLM agents, finding that malicious agents can collaborate effectively and adapt to interventions, with a benchmark covering 28 fraud scenarios.
In this work, we study the risks of collective financial fraud in large-scale multi-agent systems powered by large language model (LLM) agents. We investigate whether agents can collaborate in fraudulent behaviors, how such collaboration amplifies risks, and what factors influence fraud success. To support this research, we present MultiAgentFraudBench, a large-scale benchmark for simulating financial fraud scenarios based on realistic online interactions. The benchmark covers 28 typical online fraud scenarios, spanning the full fraud lifecycle across both public and private domains. We further analyze key factors affecting fraud success, including interaction depth, activity level, and fine-grained collaboration failure modes. Finally, we propose a series of mitigation strategies, including adding content-level warnings to fraudulent posts and dialogues, using LLMs as monitors to block potentially malicious agents, and fostering group resilience through information sharing at the societal level. Notably, we observe that malicious agents can adapt to environmental interventions. Our findings highlight the real-world risks of multi-agent financial fraud and suggest practical measures for mitigating them. Code is available at https://github.com/zheng977/MutiAgent4Fraud.