Learning not to Regret
This work addresses the challenge of efficiently solving game-theoretic equilibria in real-world scenarios with similar but non-identical games, offering a meta-learning approach that is incremental over existing regret minimization techniques.
The paper tackles the problem of accelerating equilibrium finding for distributions of similar games, such as poker with varying public cards, by meta-learning a regret minimizer tailored to the distribution. The result is Neural Predictive Regret Matching, which achieves more than tenfold faster convergence compared to non-meta-learned methods in experiments on river poker games.
The literature on game-theoretic equilibrium finding predominantly focuses on single games or their repeated play. Nevertheless, numerous real-world scenarios feature playing a game sampled from a distribution of similar, but not identical games, such as playing poker with different public cards or trading correlated assets on the stock market. As these similar games feature similar equilibra, we investigate a way to accelerate equilibrium finding on such a distribution. We present a novel "learning not to regret" framework, enabling us to meta-learn a regret minimizer tailored to a specific distribution. Our key contribution, Neural Predictive Regret Matching, is uniquely meta-learned to converge rapidly for the chosen distribution of games, while having regret minimization guarantees on any game. We validated our algorithms' faster convergence on a distribution of river poker games. Our experiments show that the meta-learned algorithms outpace their non-meta-learned counterparts, achieving more than tenfold improvements.