Using Reinforcement Learning to Validate Empirical Game-Theoretic Analysis: A Continuous Double Auction Study
This work addresses the validation challenge for simulation-based methods in strategic environments like stock markets, offering a tool to assess equilibrium stability, though it is incremental as it builds on prior EGTA and reinforcement learning techniques.
The paper tackled the problem of validating Nash-equilibrium strategy profiles found by empirical game-theoretic analysis (EGTA) in continuous double auction markets, by proposing a reinforcement learning approach to analyze regret, and provided evidence that the equilibria have negligible regret.
Empirical game-theoretic analysis (EGTA) has recently been applied successfully to analyze the behavior of large numbers of competing traders in a continuous double auction market. Multiagent simulation methods like EGTA are useful for studying complex strategic environments like a stock market, where it is not feasible to solve analytically for the rational behavior of each agent. A weakness of simulation-based methods in strategic settings, however, is that it is typically impossible to prove that the strategy profile assigned to the simulated agents is stable, as in a Nash equilibrium. I propose using reinforcement learning to analyze the regret of supposed Nash-equilibrium strategy profiles found by EGTA. I have developed a new library of reinforcement learning tools, which I have integrated into an extended version of the market simulator from our prior work. I provide evidence for the effectiveness of our library methods, both on a suite of benchmark problems from the literature, and on non-equilibrium strategy profiles in our market environment. Finally, I use our new reinforcement learning tools to provide evidence that the equilibria found by EGTA in our recent continuous double auction study are likely to have only negligible regret, even with respect to an extended strategy space.