Modular Architecture for StarCraft II with Deep Reinforcement Learning
This addresses the challenge of building complex game AIs for StarCraft II players and researchers, but it is incremental as it builds on existing modular and reinforcement learning approaches.
The authors tackled the problem of creating a modular AI for StarCraft II by splitting responsibilities among multiple modules, achieving win rates of 94% or 87% against the built-in bot in Zerg vs. Zerg matches.
We present a novel modular architecture for StarCraft II AI. The architecture splits responsibilities between multiple modules that each control one aspect of the game, such as build-order selection or tactics. A centralized scheduler reviews macros suggested by all modules and decides their order of execution. An updater keeps track of environment changes and instantiates macros into series of executable actions. Modules in this framework can be optimized independently or jointly via human design, planning, or reinforcement learning. We apply deep reinforcement learning techniques to training two out of six modules of a modular agent with self-play, achieving 94% or 87% win rates against the "Harder" (level 5) built-in Blizzard bot in Zerg vs. Zerg matches, with or without fog-of-war.