Decoding Rewards in Competitive Games: Inverse Game Theory with Entropy Regularization
This work addresses the challenge of inferring reward functions from observed behaviors in competitive settings, which is important for applications in game theory and inverse reinforcement learning, though it is incremental as it builds on existing concepts like quantal response equilibrium.
The authors tackled the problem of estimating unknown reward functions in competitive games by developing a unified framework for reward recovery in two-player zero-sum matrix and Markov games with entropy regularization, achieving strong theoretical guarantees and practical effectiveness in numerical studies.
Estimating the unknown reward functions driving agents' behaviors is of central interest in inverse reinforcement learning and game theory. To tackle this problem, we develop a unified framework for reward function recovery in two-player zero-sum matrix games and Markov games with entropy regularization, where we aim to reconstruct the underlying reward functions given observed players' strategies and actions. This task is challenging due to the inherent ambiguity of inverse problems, the non-uniqueness of feasible rewards, and limited observational data coverage. To address these challenges, we establish the reward function's identifiability using the quantal response equilibrium (QRE) under linear assumptions. Building upon this theoretical foundation, we propose a novel algorithm to learn reward functions from observed actions. Our algorithm works in both static and dynamic settings and is adaptable to incorporate different methods, such as Maximum Likelihood Estimation (MLE). We provide strong theoretical guarantees for the reliability and sample efficiency of our algorithm. Further, we conduct extensive numerical studies to demonstrate the practical effectiveness of the proposed framework, offering new insights into decision-making in competitive environments.