Efficient Phi-Regret Minimization in Extensive-Form Games via Online Mirror Descent
This work addresses computational efficiency in learning equilibria for extensive-form games, which is crucial for applications in AI and game theory, though it is incremental as it builds on existing Φ-Hedge and OMD methods.
The paper tackles the computational intractability of converting Extensive-Form Games (EFGs) to Normal-Form Games (NFGs) for learning, showing that the Φ-Hedge algorithm can efficiently learn equilibria like Nash Equilibria, NFCCE, and EFCE in EFGs by connecting it to Online Mirror Descent (OMD) with dilated regularizers, achieving a sharp Õ(√XAT) EFCE-regret under bandit-feedback, which matches the information-theoretic lower bound.
A conceptually appealing approach for learning Extensive-Form Games (EFGs) is to convert them to Normal-Form Games (NFGs). This approach enables us to directly translate state-of-the-art techniques and analyses in NFGs to learning EFGs, but typically suffers from computational intractability due to the exponential blow-up of the game size introduced by the conversion. In this paper, we address this problem in natural and important setups for the \emph{$Φ$-Hedge} algorithm -- A generic algorithm capable of learning a large class of equilibria for NFGs. We show that $Φ$-Hedge can be directly used to learn Nash Equilibria (zero-sum settings), Normal-Form Coarse Correlated Equilibria (NFCCE), and Extensive-Form Correlated Equilibria (EFCE) in EFGs. We prove that, in those settings, the \emph{$Φ$-Hedge} algorithms are equivalent to standard Online Mirror Descent (OMD) algorithms for EFGs with suitable dilated regularizers, and run in polynomial time. This new connection further allows us to design and analyze a new class of OMD algorithms based on modifying its log-partition function. In particular, we design an improved algorithm with balancing techniques that achieves a sharp $\widetilde{\mathcal{O}}(\sqrt{XAT})$ EFCE-regret under bandit-feedback in an EFG with $X$ information sets, $A$ actions, and $T$ episodes. To our best knowledge, this is the first such rate and matches the information-theoretic lower bound.