What game are we playing? End-to-end learning in normal and extensive form games
This addresses the inverse setting in game theory for AI agents, enabling learning in games with unknown parameters, though it is incremental by building on existing game-solving methods.
The paper tackles the problem of learning unknown game parameters in normal and extensive form games through observations, proposing an end-to-end differentiable learning framework that integrates a differentiable game solver into deep network architectures, with demonstrations in poker and security game tasks.
Although recent work in AI has made great progress in solving large, zero-sum, extensive-form games, the underlying assumption in most past work is that the parameters of the game itself are known to the agents. This paper deals with the relatively under-explored but equally important "inverse" setting, where the parameters of the underlying game are not known to all agents, but must be learned through observations. We propose a differentiable, end-to-end learning framework for addressing this task. In particular, we consider a regularized version of the game, equivalent to a particular form of quantal response equilibrium, and develop 1) a primal-dual Newton method for finding such equilibrium points in both normal and extensive form games; and 2) a backpropagation method that lets us analytically compute gradients of all relevant game parameters through the solution itself. This ultimately lets us learn the game by training in an end-to-end fashion, effectively by integrating a "differentiable game solver" into the loop of larger deep network architectures. We demonstrate the effectiveness of the learning method in several settings including poker and security game tasks.