AIR: Unifying Individual and Collective Exploration in Cooperative Multi-Agent Reinforcement Learning
This addresses the problem of reduced training efficiency in MARL for researchers and practitioners, though it appears incremental as it builds on existing exploration approaches.
The paper tackles the challenge of exploration in cooperative multi-agent reinforcement learning by proposing AIR, a method that unifies individual and collective exploration through adversarial components, resulting in improved efficiency and effectiveness across various tasks.
Exploration in cooperative multi-agent reinforcement learning (MARL) remains challenging for value-based agents due to the absence of an explicit policy. Existing approaches include individual exploration based on uncertainty towards the system and collective exploration through behavioral diversity among agents. However, the introduction of additional structures often leads to reduced training efficiency and infeasible integration of these methods. In this paper, we propose Adaptive exploration via Identity Recognition~(AIR), which consists of two adversarial components: a classifier that recognizes agent identities from their trajectories, and an action selector that adaptively adjusts the mode and degree of exploration. We theoretically prove that AIR can facilitate both individual and collective exploration during training, and experiments also demonstrate the efficiency and effectiveness of AIR across various tasks.