VirtualCrime: Evaluating Criminal Potential of Large Language Models via Sandbox Simulation
This work addresses safety concerns for deploying AI agents in real-world settings by providing a method to assess criminal capabilities, though it is incremental as it builds on existing simulation and evaluation techniques.
The paper tackles the problem of evaluating the potential misuse of large language models for criminal activities by proposing VirtualCrime, a sandbox simulation framework with 40 crime tasks, and finds that all tested LLMs can generate detailed crime plans with some achieving high success rates, including cases where agents harm NPCs.
Large language models (LLMs) have shown strong capabilities in multi-step decision-making, planning and actions, and are increasingly integrated into various real-world applications. It is concerning whether their strong problem-solving abilities may be misused for crimes. To address this gap, we propose VirtualCrime, a sandbox simulation framework based on a three-agent system to evaluate the criminal capabilities of models. Specifically, this framework consists of an attacker agent acting as the leader of a criminal team, a judge agent determining the outcome of each action, and a world manager agent updating the environment state and entities. Furthermore, we design 40 diverse crime tasks within this framework, covering 11 maps and 13 crime objectives such as theft, robbery, kidnapping, and riot. We also introduce a human player baseline for reference to better interpret the performance of LLM agents. We evaluate 8 strong LLMs and find (1) All agents in the simulation environment compliantly generate detailed plans and execute intelligent crime processes, with some achieving relatively high success rates; (2) In some cases, agents take severe action that inflicts harm to NPCs to achieve their goals. Our work highlights the need for safety alignment when deploying agentic AI in real-world settings.