Mastering Strategy Card Game (Hearthstone) with Improved Techniques
This work addresses the challenge of AI decision-making in complex, real-world games for the AI and gaming communities, though it is incremental as it builds on prior methods.
The authors tackled the problem of AI mastering the complex commercial strategy card game Hearthstone by applying and improving existing algorithms, resulting in models that defeated a top-ranked human player in all Best-of-5 tournaments.
Strategy card game is a well-known genre that is demanding on the intelligent game-play and can be an ideal test-bench for AI. Previous work combines an end-to-end policy function and an optimistic smooth fictitious play, which shows promising performances on the strategy card game Legend of Code and Magic. In this work, we apply such algorithms to Hearthstone, a famous commercial game that is more complicated in game rules and mechanisms. We further propose several improved techniques and consequently achieve significant progress. For a machine-vs-human test we invite a Hearthstone streamer whose best rank was top 10 of the official league in China region that is estimated to be of millions of players. Our models defeat the human player in all Best-of-5 tournaments of full games (including both deck building and battle), showing a strong capability of decision making.