Center of Gravity-Guided Focusing Influence Mechanism for Multi-Agent Reinforcement Learning
This addresses the problem of limited exploration and coordination in multi-agent systems for researchers, though it appears incremental as it builds on existing MARL methods with a novel mechanism.
The paper tackles the challenge of cooperative multi-agent reinforcement learning under sparse rewards by proposing the Focusing Influence Mechanism (FIM), which directs agent influence toward critical state dimensions, resulting in significantly improved cooperative performance across benchmarks.
Cooperative multi-agent reinforcement learning (MARL) under sparse rewards presents a fundamental challenge due to limited exploration and insufficient coordinated attention among agents. In this work, we propose the Focusing Influence Mechanism (FIM), a novel framework that enhances cooperation by directing agent influence toward task-critical elements, referred to as Center of Gravity (CoG) state dimensions, inspired by Clausewitz's military theory. FIM consists of three core components: (1) identifying CoG state dimensions based on their stability under agent behavior, (2) designing counterfactual intrinsic rewards to promote meaningful influence on these dimensions, and (3) encouraging persistent and synchronized focus through eligibility-trace-based credit accumulation. These mechanisms enable agents to induce more targeted and effective state transitions, facilitating robust cooperation even in extremely sparse reward settings. Empirical evaluations across diverse MARL benchmarks demonstrate that the proposed FIM significantly improves cooperative performance compared to baselines.