ROMAX: Certifiably Robust Deep Multiagent Reinforcement Learning via Convex Relaxation
This addresses robustness challenges in multirobot systems for applications like autonomous vehicles or drones, though it appears incremental as it builds on existing minimax and convex relaxation techniques.
The paper tackles the robustness issue in multiagent reinforcement learning against cyber-physical attacks by proposing a minimax approach with convex relaxation to infer worst-case policy updates, achieving a certified bound and outperforming previous state-of-the-art methods on mixed cooperative-competitive tasks.
In a multirobot system, a number of cyber-physical attacks (e.g., communication hijack, observation perturbations) can challenge the robustness of agents. This robustness issue worsens in multiagent reinforcement learning because there exists the non-stationarity of the environment caused by simultaneously learning agents whose changing policies affect the transition and reward functions. In this paper, we propose a minimax MARL approach to infer the worst-case policy update of other agents. As the minimax formulation is computationally intractable to solve, we apply the convex relaxation of neural networks to solve the inner minimization problem. Such convex relaxation enables robustness in interacting with peer agents that may have significantly different behaviors and also achieves a certified bound of the original optimization problem. We evaluate our approach on multiple mixed cooperative-competitive tasks and show that our method outperforms the previous state of the art approaches on this topic.