Similarity-based cooperative equilibrium
This addresses the challenge of fostering cooperation among autonomous ML agents in realistic, partially transparent environments, representing an incremental advance over prior full transparency approaches.
The paper tackles the problem of enabling cooperation between machine learning agents in one-shot social dilemmas like the Prisoner's Dilemma, where full transparency is unrealistic, by introducing a setting where agents observe only a similarity score; it proves this allows for cooperative outcomes equivalent to full transparency and demonstrates that cooperation can be learned using simple ML methods.
As machine learning agents act more autonomously in the world, they will increasingly interact with each other. Unfortunately, in many social dilemmas like the one-shot Prisoner's Dilemma, standard game theory predicts that ML agents will fail to cooperate with each other. Prior work has shown that one way to enable cooperative outcomes in the one-shot Prisoner's Dilemma is to make the agents mutually transparent to each other, i.e., to allow them to access one another's source code (Rubinstein 1998, Tennenholtz 2004) -- or weights in the case of ML agents. However, full transparency is often unrealistic, whereas partial transparency is commonplace. Moreover, it is challenging for agents to learn their way to cooperation in the full transparency setting. In this paper, we introduce a more realistic setting in which agents only observe a single number indicating how similar they are to each other. We prove that this allows for the same set of cooperative outcomes as the full transparency setting. We also demonstrate experimentally that cooperation can be learned using simple ML methods.