Safe Multiagent Coordination via Entropic Exploration
This addresses safety concerns in multiagent coordination for real-world applications, though it appears incremental by focusing on team constraints rather than individual ones.
The paper tackles the problem of limited exploration in safe multiagent reinforcement learning due to safety constraints, proposing entropic exploration for constrained multiagent reinforcement learning (E2C) that uses observation entropy maximization. Experiments show E2C matches or surpasses baselines in task performance while reducing unsafe behaviors by up to 50%.
Many real-world multiagent learning problems involve safety concerns. In these setups, typical safe reinforcement learning algorithms constrain agents' behavior, limiting exploration -- a crucial component for discovering effective cooperative multiagent behaviors. Moreover, the multiagent literature typically models individual constraints for each agent and has yet to investigate the benefits of using joint team constraints. In this work, we analyze these team constraints from a theoretical and practical perspective and propose entropic exploration for constrained multiagent reinforcement learning (E2C) to address the exploration issue. E2C leverages observation entropy maximization to incentivize exploration and facilitate learning safe and effective cooperative behaviors. Experiments across increasingly complex domains show that E2C agents match or surpass common unconstrained and constrained baselines in task performance while reducing unsafe behaviors by up to $50\%$.