Bayes' Bluff: Opponent Modelling in Poker
This addresses the challenge of AI in poker with partial observability and unknown adversaries, but is incremental as it builds on existing Bayesian methods.
The paper tackles the problem of opponent modeling in poker by presenting a Bayesian probabilistic model that separates game dynamics from opponent strategy uncertainty, and demonstrates effective responses in reduced poker and Texas hold'em.
Poker is a challenging problem for artificial intelligence, with non-deterministic dynamics, partial observability, and the added difficulty of unknown adversaries. Modelling all of the uncertainties in this domain is not an easy task. In this paper we present a Bayesian probabilistic model for a broad class of poker games, separating the uncertainty in the game dynamics from the uncertainty of the opponent's strategy. We then describe approaches to two key subproblems: (i) inferring a posterior over opponent strategies given a prior distribution and observations of their play, and (ii) playing an appropriate response to that distribution. We demonstrate the overall approach on a reduced version of poker using Dirichlet priors and then on the full game of Texas hold'em using a more informed prior. We demonstrate methods for playing effective responses to the opponent, based on the posterior.