Robustness to Multi-Modal Environment Uncertainty in MARL using Curriculum Learning
This addresses the challenge of deploying MARL in real-world scenarios where multiple uncertainties occur simultaneously, representing an incremental advance over prior work focused on single-variable uncertainty.
The paper tackles the problem of making multi-agent reinforcement learning policies robust to simultaneous uncertainties in multiple environment variables, proposing a curriculum learning approach that achieves state-of-the-art robustness in cooperative and competitive settings.
Multi-agent reinforcement learning (MARL) plays a pivotal role in tackling real-world challenges. However, the seamless transition of trained policies from simulations to real-world requires it to be robust to various environmental uncertainties. Existing works focus on finding Nash Equilibrium or the optimal policy under uncertainty in one environment variable (i.e. action, state or reward). This is because a multi-agent system itself is highly complex and unstationary. However, in real-world situation uncertainty can occur in multiple environment variables simultaneously. This work is the first to formulate the generalised problem of robustness to multi-modal environment uncertainty in MARL. To this end, we propose a general robust training approach for multi-modal uncertainty based on curriculum learning techniques. We handle two distinct environmental uncertainty simultaneously and present extensive results across both cooperative and competitive MARL environments, demonstrating that our approach achieves state-of-the-art levels of robustness.