Deep Multi-Agent Reinforcement Learning with Hybrid Action Spaces based on Maximum Entropy
This addresses a gap in multi-agent systems for real-world applications with hybrid action spaces, though it is incremental as it extends existing methods.
The paper tackled the problem of multi-agent reinforcement learning in environments with hybrid action spaces, proposing the MAHSAC algorithm which showed good performance in training speed, stability, and anti-interference ability, outperforming existing methods in cooperative and competitive scenarios.
Multi-agent deep reinforcement learning has been applied to address a variety of complex problems with either discrete or continuous action spaces and achieved great success. However, most real-world environments cannot be described by only discrete action spaces or only continuous action spaces. And there are few works having ever utilized deep reinforcement learning (drl) to multi-agent problems with hybrid action spaces. Therefore, we propose a novel algorithm: Deep Multi-Agent Hybrid Soft Actor-Critic (MAHSAC) to fill this gap. This algorithm follows the centralized training but decentralized execution (CTDE) paradigm, and extend the Soft Actor-Critic algorithm (SAC) to handle hybrid action space problems in Multi-Agent environments based on maximum entropy. Our experiences are running on an easy multi-agent particle world with a continuous observation and discrete action space, along with some basic simulated physics. The experimental results show that MAHSAC has good performance in training speed, stability, and anti-interference ability. At the same time, it outperforms existing independent deep hybrid learning method in cooperative scenarios and competitive scenarios.