When Ethics and Payoffs Diverge: LLM Agents in Morally Charged Social Dilemmas
This addresses AI safety concerns for developers and users by revealing ethical misalignment in LLMs when incentives conflict with morals, though it is incremental as it builds on prior work on moral judgment and strategic behavior.
The study investigated how large language models (LLMs) behave in social dilemmas where moral imperatives conflict with rewards, using a simulation called MoralSim. Results showed substantial variation across models, with no model consistently acting morally, highlighting risks in deploying LLMs in agentic roles.
Recent advances in large language models (LLMs) have enabled their use in complex agentic roles, involving decision-making with humans or other agents, making ethical alignment a key AI safety concern. While prior work has examined both LLMs' moral judgment and strategic behavior in social dilemmas, there is limited understanding of how they act when moral imperatives directly conflict with rewards or incentives. To investigate this, we introduce Moral Behavior in Social Dilemma Simulation (MoralSim) and evaluate how LLMs behave in the prisoner's dilemma and public goods game with morally charged contexts. In MoralSim, we test a range of frontier models across both game structures and three distinct moral framings, enabling a systematic examination of how LLMs navigate social dilemmas in which ethical norms conflict with payoff-maximizing strategies. Our results show substantial variation across models in both their general tendency to act morally and the consistency of their behavior across game types, the specific moral framing, and situational factors such as opponent behavior and survival risks. Crucially, no model exhibits consistently moral behavior in MoralSim, highlighting the need for caution when deploying LLMs in agentic roles where the agent's "self-interest" may conflict with ethical expectations. Our code is available at https://github.com/sbackmann/moralsim.