Supervised and Reinforcement Learning from Observations in Reconnaissance Blind Chess
This work addresses the challenge of playing imperfect information games for AI researchers, but it is incremental as it adapts existing methods to a specific domain.
The authors tackled the problem of playing Reconnaissance Blind Chess, an imperfect information game, by training an agent using supervised learning on game records followed by self-play reinforcement learning without search, achieving an ELO of 1330 and ranking 27th on the leaderboard.
In this work, we adapt a training approach inspired by the original AlphaGo system to play the imperfect information game of Reconnaissance Blind Chess. Using only the observations instead of a full description of the game state, we first train a supervised agent on publicly available game records. Next, we increase the performance of the agent through self-play with the on-policy reinforcement learning algorithm Proximal Policy Optimization. We do not use any search to avoid problems caused by the partial observability of game states and only use the policy network to generate moves when playing. With this approach, we achieve an ELO of 1330 on the RBC leaderboard, which places our agent at position 27 at the time of this writing. We see that self-play significantly improves performance and that the agent plays acceptably well without search and without making assumptions about the true game state.