The AI Economist: Optimal Economic Policy Design via Two-level Deep Reinforcement Learning
This work addresses economic policy design for policymakers and economists by providing a computational learning-based approach to overcome limitations in traditional methods, though it is incremental in applying RL to a new domain.
The paper tackles the challenge of designing optimal economic policies by introducing the AI Economist, a two-level deep reinforcement learning framework that trains agents and a social planner to co-adapt. It demonstrates efficacy in optimal taxation, recovering theoretical policies in simple economies and improving social welfare and equality-productivity trade-offs in complex ones, with concrete gains over baselines.
AI and reinforcement learning (RL) have improved many areas, but are not yet widely adopted in economic policy design, mechanism design, or economics at large. At the same time, current economic methodology is limited by a lack of counterfactual data, simplistic behavioral models, and limited opportunities to experiment with policies and evaluate behavioral responses. Here we show that machine-learning-based economic simulation is a powerful policy and mechanism design framework to overcome these limitations. The AI Economist is a two-level, deep RL framework that trains both agents and a social planner who co-adapt, providing a tractable solution to the highly unstable and novel two-level RL challenge. From a simple specification of an economy, we learn rational agent behaviors that adapt to learned planner policies and vice versa. We demonstrate the efficacy of the AI Economist on the problem of optimal taxation. In simple one-step economies, the AI Economist recovers the optimal tax policy of economic theory. In complex, dynamic economies, the AI Economist substantially improves both utilitarian social welfare and the trade-off between equality and productivity over baselines. It does so despite emergent tax-gaming strategies, while accounting for agent interactions and behavioral change more accurately than economic theory. These results demonstrate for the first time that two-level, deep RL can be used for understanding and as a complement to theory for economic design, unlocking a new computational learning-based approach to understanding economic policy.