Game Theory with Simulation in the Presence of Unpredictable Randomisation
This work addresses game theory challenges for AI systems, but it is incremental as it builds on prior pure-strategy simulation research.
The paper tackles the problem of leveraging AI agent predictability to improve social welfare in game theory by studying mixed-strategy simulation, showing negative results like NP-hardness in finding Pareto-improving equilibria and positive results where it can enhance welfare under specific conditions like scalable trust or privacy needs.
AI agents will be predictable in certain ways that traditional agents are not. Where and how can we leverage this predictability in order to improve social welfare? We study this question in a game-theoretic setting where one agent can pay a fixed cost to simulate the other in order to learn its mixed strategy. As a negative result, we prove that, in contrast to prior work on pure-strategy simulation, enabling mixed-strategy simulation may no longer lead to improved outcomes for both players in all so-called "generalised trust games". In fact, mixed-strategy simulation does not help in any game where the simulatee's action can depend on that of the simulator. We also show that, in general, deciding whether simulation introduces Pareto-improving Nash equilibria in a given game is NP-hard. As positive results, we establish that mixed-strategy simulation can improve social welfare if the simulator has the option to scale their level of trust, if the players face challenges with both trust and coordination, or if maintaining some level of privacy is essential for enabling cooperation.