Towards Learning Representations of Policies in Two-Player Zero-Sum Imperfect-Information Games
This work provides an early systematic comparison of self-supervised learning for policy embeddings in games, but the methods are basic and results are preliminary.
The paper investigates learning policy representations in two-player zero-sum imperfect-information games, introducing dataset creation methods, representation learning techniques, and downstream tasks. Evaluations on Kuhn and Leduc Poker show that basic methods can learn useful behavioral representations.
We investigate the problem of learning useful policy representations (embeddings) in two-player zero-sum imperfect-information games. We make three contributions: First, we introduce methods of creating datasets of policies for a given game. Second, we propose methods to learn policy representations. Third, we introduce downstream tasks to evaluate the effectiveness of such representations. We evaluate each dataset method, embedding method, and downstream task on Kuhn and Leduc Poker. Although our methods are very basic, we demonstrate that useful behavioral representations are present in the learned embeddings. To our knowledge, this work is among the first to systematically compare self-supervised learning techniques for learning policy representations in games. Our code is available at https://github.com/VitamintK/ssl-project for others to extend.