Scalable Safe Multi-Agent Reinforcement Learning for Multi-Agent System
This addresses safety and scalability issues for multi-agent reinforcement learning systems, representing an incremental improvement over existing methods.
The paper tackled the challenges of safety and scalability in Multi-Agent Systems by proposing SS-MARL, a framework that uses a multi-layer message passing network and constrained joint policy optimization, achieving a better trade-off between optimality and safety and significantly outperforming baselines in scalability with large numbers of agents.
Safety and scalability are two critical challenges faced by practical Multi-Agent Systems (MAS). However, existing Multi-Agent Reinforcement Learning (MARL) algorithms that rely solely on reward shaping are ineffective in ensuring safety, and their scalability is rather limited due to the fixed-size network output. To address these issues, we propose a novel framework, Scalable Safe MARL (SS-MARL), to enhance the safety and scalability of MARL methods. Leveraging the inherent graph structure of MAS, we design a multi-layer message passing network to aggregate local observations and communications of varying sizes. Furthermore, we develop a constrained joint policy optimization method in the setting of local observation to improve safety. Simulation experiments demonstrate that SS-MARL achieves a better trade-off between optimality and safety compared to baselines, and its scalability significantly outperforms the latest methods in scenarios with a large number of agents.