Accelerated regularized learning in finite N-person games
This work provides a significant speed-up for online learning in game theory, applicable to scenarios from full information to bandit feedback, though it is incremental in building on Nesterov's acceleration.
The authors tackled the problem of accelerating convergence to Nash equilibria in finite N-person games by introducing the FTXL method, which incorporates momentum into regularized learning, achieving superlinear convergence rates across various feedback structures.
Motivated by the success of Nesterov's accelerated gradient algorithm for convex minimization problems, we examine whether it is possible to achieve similar performance gains in the context of online learning in games. To that end, we introduce a family of accelerated learning methods, which we call "follow the accelerated leader" (FTXL), and which incorporates the use of momentum within the general framework of regularized learning - and, in particular, the exponential/multiplicative weights algorithm and its variants. Drawing inspiration and techniques from the continuous-time analysis of Nesterov's algorithm, we show that FTXL converges locally to strict Nash equilibria at a superlinear rate, achieving in this way an exponential speed-up over vanilla regularized learning methods (which, by comparison, converge to strict equilibria at a geometric, linear rate). Importantly, FTXL maintains its superlinear convergence rate in a broad range of feedback structures, from deterministic, full information models to stochastic, realization-based ones, and even when run with bandit, payoff-based information, where players are only able to observe their individual realized payoffs.