Learning and Improving Backgammon Strategy
This addresses the problem of improving AI strategy in backgammon for game-playing systems, representing an incremental advance in reinforcement learning techniques.
The paper tackled learning a backgammon value function by combining online and offline methods, including parallel neural network training and TD(λ) reinforcement learning with Monte-Carlo rollouts, achieving performance comparable to or better than top human and computer players.
A novel approach to learning is presented, combining features of on-line and off-line methods to achieve considerable performance in the task of learning a backgammon value function in a process that exploits the processing power of parallel supercomputers. The off-line methods comprise a set of techniques for parallelizing neural network training and $TD(λ)$ reinforcement learning; here Monte-Carlo ``Rollouts'' are introduced as a massively parallel on-line policy improvement technique which applies resources to the decision points encountered during the search of the game tree to further augment the learned value function estimate. A level of play roughly as good as, or possibly better than, the current champion human and computer backgammon players has been achieved in a short period of learning.