Contract-Based Compositional Shielding for Safe Multi-Agent Reinforcement Learning
For multi-agent reinforcement learning systems requiring decentralized execution and safety guarantees, this work provides a method to achieve team-optimal coordination without sacrificing safety, addressing a key bottleneck in safe multi-agent coordination.
The paper introduces a contract-based compositional shielding method for multi-agent reinforcement learning that ensures global safety without centralized runtime control, achieving team-optimal safe behavior. In evaluations across 6 environments and 15 algorithmic variants, the approach recovers near-optimal performance while maintaining safety guarantees.
Safe coordination problems surface in multi-agent reinforcement learning when global safety cannot be enforced by any agent unilaterally: the admissibility of one agent's action may depend on the dynamics of other agents. Decentralised shields can enforce safety at runtime, but purely factorised permissions often exclude optimal team behaviour that is safe only through coordination. We study deterministic safety guarantees for agents trained and deployed under decentralised execution, recovering team-optimal safe behaviour without centralised runtime control. Agents have a shared global specification $φ$ in the safety fragment of Linear Temporal Logic ($\mathsf{LTL}_{\mathsf{safe}}$ ), and select among tuples of local $\mathsf{LTL}_{\mathsf{safe}}$ obligations whose conjunction implies the global specification $φ$. Each agent may rely on the other agents' local obligations as assumptions because the whole contract tuple is certified simultaneously and allows projection into local action masks. At learning time, a non-stationary multi-armed bandit chooses among a library of local $\mathsf{LTL}_{\mathsf{safe}}$ obligations to select the tuple that optimises team reward, all without forgoing end-to-end safety. We evaluate the approach across 6 environments and 15 algorithmic variants.