MAIDCRL: Semi-centralized Multi-Agent Influence Dense-CNN Reinforcement Learning
This work addresses multi-agent reinforcement learning for cooperative and competitive systems, but it is incremental as it builds on an existing method with architectural enhancements.
The authors tackled the challenge of distributed decision-making in multi-agent systems by extending MAIDRL with convolutional layers to create MAIDCRL, which significantly improved learning performance and achieved a faster learning rate, especially on complex heterogeneous SMAC scenarios.
Distributed decision-making in multi-agent systems presents difficult challenges for interactive behavior learning in both cooperative and competitive systems. To mitigate this complexity, MAIDRL presents a semi-centralized Dense Reinforcement Learning algorithm enhanced by agent influence maps (AIMs), for learning effective multi-agent control on StarCraft Multi-Agent Challenge (SMAC) scenarios. In this paper, we extend the DenseNet in MAIDRL and introduce semi-centralized Multi-Agent Dense-CNN Reinforcement Learning, MAIDCRL, by incorporating convolutional layers into the deep model architecture, and evaluate the performance on both homogeneous and heterogeneous scenarios. The results show that the CNN-enabled MAIDCRL significantly improved the learning performance and achieved a faster learning rate compared to the existing MAIDRL, especially on more complicated heterogeneous SMAC scenarios. We further investigate the stability and robustness of our model. The statistics reflect that our model not only achieves higher winning rate in all the given scenarios but also boosts the agent's learning process in fine-grained decision-making.