Position: Emergent Machina Sapiens Urge Rethinking Multi-Agent Paradigms
This addresses the problem of potential chaos and safety risks in critical infrastructure for stakeholders deploying AI systems, but it is incremental as it builds on existing multi-agent paradigms.
The paper tackles the challenge of uncoordinated AI systems with misaligned objectives coexisting in shared critical infrastructure domains, advocating for a fundamental rethinking of multi-agent frameworks to enable dynamic objective adjustment and emergent self-organization.
Artificial Intelligence (AI) agents capable of autonomous learning and independent decision-making hold great promise for addressing complex challenges across various critical infrastructure domains, including transportation, energy systems, and manufacturing. However, the surge in the design and deployment of AI systems, driven by various stakeholders with distinct and unaligned objectives, introduces a crucial challenge: How can uncoordinated AI systems coexist and evolve harmoniously in shared environments without creating chaos or compromising safety? To address this, we advocate for a fundamental rethinking of existing multi-agent frameworks, such as multi-agent systems and game theory, which are largely limited to predefined rules and static objective structures. We posit that AI agents should be empowered to adjust their objectives dynamically, make compromises, form coalitions, and safely compete or cooperate through evolving relationships and social feedback. Through two case studies in critical infrastructure applications, we call for a shift toward the emergent, self-organizing, and context-aware nature of these multi-agentic AI systems.