Novelty-Guided Data Reuse for Efficient and Diversified Multi-Agent Reinforcement Learning
This work addresses efficiency and diversity issues in multi-agent reinforcement learning for AI systems in collaborative environments, representing an incremental advancement.
The paper tackled the problem of inefficient sample utilization and lack of diversity in multi-agent reinforcement learning by introducing a novelty-guided sample reuse method, resulting in substantial improvements in complex cooperative tasks like Google Research Football and StarCraft II.
Recently, deep Multi-Agent Reinforcement Learning (MARL) has demonstrated its potential to tackle complex cooperative tasks, pushing the boundaries of AI in collaborative environments. However, the efficiency of these systems is often compromised by inadequate sample utilization and a lack of diversity in learning strategies. To enhance MARL performance, we introduce a novel sample reuse approach that dynamically adjusts policy updates based on observation novelty. Specifically, we employ a Random Network Distillation (RND) network to gauge the novelty of each agent's current state, assigning additional sample update opportunities based on the uniqueness of the data. We name our method Multi-Agent Novelty-GuidEd sample Reuse (MANGER). This method increases sample efficiency and promotes exploration and diverse agent behaviors. Our evaluations confirm substantial improvements in MARL effectiveness in complex cooperative scenarios such as Google Research Football and super-hard StarCraft II micromanagement tasks.