The Coming Crisis of Multi-Agent Misalignment: AI Alignment Must Be a Dynamic and Social Process
This addresses the problem of AI misalignment in multi-agent systems for researchers and practitioners, highlighting an incremental perspective by applying social science insights to an existing challenge.
The paper argues that AI alignment in multi-agent systems must be treated as a dynamic, social process dependent on interactions, as social structures can unintentionally misalign agents with human values, and it calls for new simulation environments and benchmarks to assess alignment in these contexts.
This position paper states that AI Alignment in Multi-Agent Systems (MAS) should be considered a dynamic and interaction-dependent process that heavily depends on the social environment where agents are deployed, either collaborative, cooperative, or competitive. While AI alignment with human values and preferences remains a core challenge, the growing prevalence of MAS in real-world applications introduces a new dynamic that reshapes how agents pursue goals and interact to accomplish various tasks. As agents engage with one another, they must coordinate to accomplish both individual and collective goals. However, this complex social organization may unintentionally misalign some or all of these agents with human values or user preferences. Drawing on social sciences, we analyze how social structure can deter or shatter group and individual values. Based on these analyses, we call on the AI community to treat human, preferential, and objective alignment as an interdependent concept, rather than isolated problems. Finally, we emphasize the urgent need for simulation environments, benchmarks, and evaluation frameworks that allow researchers to assess alignment in these interactive multi-agent contexts before such dynamics grow too complex to control.