From Game-theoretic Multi-agent Log Linear Learning to Reinforcement Learning
This work addresses multi-agent learning challenges for researchers in game theory and reinforcement learning, offering incremental improvements to existing algorithms.
The paper tackles the limitations of game-theoretic log-linear learning by introducing a variant that relaxes structural assumptions, and enhances reinforcement learning with a double-aggregation scheme and constant step-size to improve convergence rates, as verified through numerical experiments.
The main focus of this paper is on enhancement of two types of game-theoretic learning algorithms: log-linear learning and reinforcement learning. The standard analysis of log-linear learning needs a highly structured environment, i.e. strong assumptions about the game from an implementation perspective. In this paper, we introduce a variant of log-linear learning that provides asymptotic guarantees while relaxing the structural assumptions to include synchronous updates and limitations in information available to the players. On the other hand, model-free reinforcement learning is able to perform even under weaker assumptions on players' knowledge about the environment and other players' strategies. We propose a reinforcement algorithm that uses a double-aggregation scheme in order to deepen players' insight about the environment and constant learning step-size which achieves a higher convergence rate. Numerical experiments are conducted to verify each algorithm's robustness and performance.