Mastering Strategy Card Game (Legends of Code and Magic) via End-to-End Policy and Optimistic Smooth Fictitious Play
This work addresses multi-stage decision-making problems in games, which is incremental as it builds on existing deep reinforcement learning and fictitious play methods.
The authors tackled the challenge of multi-stage games with varying observation and action spaces by studying the two-stage strategy card game Legends of Code and Magic, proposing an end-to-end policy and an optimistic smooth fictitious play algorithm, which won double championships in the COG2022 competition.
Deep Reinforcement Learning combined with Fictitious Play shows impressive results on many benchmark games, most of which are, however, single-stage. In contrast, real-world decision making problems may consist of multiple stages, where the observation spaces and the action spaces can be completely different across stages. We study a two-stage strategy card game Legends of Code and Magic and propose an end-to-end policy to address the difficulties that arise in multi-stage game. We also propose an optimistic smooth fictitious play algorithm to find the Nash Equilibrium for the two-player game. Our approach wins double championships of COG2022 competition. Extensive studies verify and show the advancement of our approach.