Learning Diverse Policies in MOBA Games via Macro-Goals
This addresses the problem of limited strategic variety in game-playing AI for MOBA games, representing an incremental improvement over existing methods.
The paper tackles the lack of policy diversity in AI systems for MOBA games by proposing the Macro-Goals Guided (MGG) framework, which learns diverse policies and outperforms state-of-the-art methods on 102 heroes in Honor of Kings.
Recently, many researchers have made successful progress in building the AI systems for MOBA-game-playing with deep reinforcement learning, such as on Dota 2 and Honor of Kings. Even though these AI systems have achieved or even exceeded human-level performance, they still suffer from the lack of policy diversity. In this paper, we propose a novel Macro-Goals Guided framework, called MGG, to learn diverse policies in MOBA games. MGG abstracts strategies as macro-goals from human demonstrations and trains a Meta-Controller to predict these macro-goals. To enhance policy diversity, MGG samples macro-goals from the Meta-Controller prediction and guides the training process towards these goals. Experimental results on the typical MOBA game Honor of Kings demonstrate that MGG can execute diverse policies in different matches and lineups, and also outperform the state-of-the-art methods over 102 heroes.