ExIt-OOS: Towards Learning from Planning in Imperfect Information Games
This work addresses the problem of developing AI agents for imperfect information games, which is incremental as it extends existing perfect information game methods.
The authors tackled the challenge of applying planning and deep reinforcement learning to imperfect information games by introducing ExIt-OOS, which integrates Online Outcome Sampling into the Expert Iteration framework, achieving a learning and planning feedback loop.
The current state of the art in playing many important perfect information games, including Chess and Go, combines planning and deep reinforcement learning with self-play. We extend this approach to imperfect information games and present ExIt-OOS, a novel approach to playing imperfect information games within the Expert Iteration framework and inspired by AlphaZero. We use Online Outcome Sampling, an online search algorithm for imperfect information games in place of MCTS. While training online, our neural strategy is used to improve the accuracy of playouts in OOS, allowing a learning and planning feedback loop for imperfect information games.