The Good Shepherd: An Oracle Agent for Mechanism Design
This addresses the challenge for institutions in managing adaptive constituents, offering a method to influence behaviors automatically, though it appears incremental in extending multiagent learning to dynamic environments.
The paper tackles the problem of designing artificial agents (mechanisms) that perform well when interacting with adaptive, learning co-players, rather than fixed ones, by proposing a two-loop learning algorithm. The results show these mechanisms can shepherd participants' strategies toward favorable outcomes, with performance demonstrated in simulations involving both artificial agents and humans.
From social networks to traffic routing, artificial learning agents are playing a central role in modern institutions. We must therefore understand how to leverage these systems to foster outcomes and behaviors that align with our own values and aspirations. While multiagent learning has received considerable attention in recent years, artificial agents have been primarily evaluated when interacting with fixed, non-learning co-players. While this evaluation scheme has merit, it fails to capture the dynamics faced by institutions that must deal with adaptive and continually learning constituents. Here we address this limitation, and construct agents ("mechanisms") that perform well when evaluated over the learning trajectory of their adaptive co-players ("participants"). The algorithm we propose consists of two nested learning loops: an inner loop where participants learn to best respond to fixed mechanisms; and an outer loop where the mechanism agent updates its policy based on experience. We report the performance of our mechanism agents when paired with both artificial learning agents and humans as co-players. Our results show that our mechanisms are able to shepherd the participants strategies towards favorable outcomes, indicating a path for modern institutions to effectively and automatically influence the strategies and behaviors of their constituents.