Final Adaptation Reinforcement Learning for N-Player Games
This work addresses the challenge of scaling reinforcement learning to multi-player games, which is incremental as it builds on existing TD-, SARSA-, and Q-learning methods.
The paper tackles the problem of applying reinforcement learning to n-player games by introducing Final Adaptation RL (FARL), a player-centered approach that propagates rewards across rounds, and reports that FARL is crucial for learning near-perfect strategies in most tested board games like Othello and ConnectFour.
This paper covers n-tuple-based reinforcement learning (RL) algorithms for games. We present new algorithms for TD-, SARSA- and Q-learning which work seamlessly on various games with arbitrary number of players. This is achieved by taking a player-centered view where each player propagates his/her rewards back to previous rounds. We add a new element called Final Adaptation RL (FARL) to all these algorithms. Our main contribution is that FARL is a vitally important ingredient to achieve success with the player-centered view in various games. We report results on seven board games with 1, 2 and 3 players, including Othello, ConnectFour and Hex. In most cases it is found that FARL is important to learn a near-perfect playing strategy. All algorithms are available in the GBG framework on GitHub.