DanZero: Mastering GuanDan Game with Reinforcement Learning
This work solves a specific problem for AI researchers and developers by advancing card game AI, but it is incremental as it applies existing reinforcement learning techniques to a new, more complex game.
The authors tackled the challenge of developing an AI for the complex card game GuanDan, which has large state and action spaces, long episodes, and an unsure number of players, by proposing DanZero, a reinforcement learning-based program that outperformed 8 baseline AI programs and achieved human-level performance after training for 30 days using 160 CPUs and 1 GPU.
Card game AI has always been a hot topic in the research of artificial intelligence. In recent years, complex card games such as Mahjong, DouDizhu and Texas Hold'em have been solved and the corresponding AI programs have reached the level of human experts. In this paper, we are devoted to developing an AI program for a more complex card game, GuanDan, whose rules are similar to DouDizhu but much more complicated. To be specific, the characteristics of large state and action space, long length of one episode and the unsure number of players in the GuanDan pose great challenges for the development of the AI program. To address these issues, we propose the first AI program DanZero for GuanDan using reinforcement learning technique. Specifically, we utilize a distributed framework to train our AI system. In the actor processes, we carefully design the state features and agents generate samples by self-play. In the learner process, the model is updated by Deep Monte-Carlo Method. After training for 30 days using 160 CPUs and 1 GPU, we get our DanZero bot. We compare it with 8 baseline AI programs which are based on heuristic rules and the results reveal the outstanding performance of DanZero. We also test DanZero with human players and demonstrate its human-level performance.