Learning Reciprocity in Complex Sequential Social Dilemmas
This addresses the challenge of extending simple reciprocity strategies like tit-for-tat to complex, multi-agent environments, though it is incremental in adapting existing reinforcement learning methods.
The authors tackled the problem of applying reciprocity to real-world sequential social dilemmas by developing an online reinforcement learning algorithm that exhibits reciprocal behavior, showing it can promote pro-social outcomes in both 2-player and 5-player scenarios.
Reciprocity is an important feature of human social interaction and underpins our cooperative nature. What is more, simple forms of reciprocity have proved remarkably resilient in matrix game social dilemmas. Most famously, the tit-for-tat strategy performs very well in tournaments of Prisoner's Dilemma. Unfortunately this strategy is not readily applicable to the real world, in which options to cooperate or defect are temporally and spatially extended. Here, we present a general online reinforcement learning algorithm that displays reciprocal behavior towards its co-players. We show that it can induce pro-social outcomes for the wider group when learning alongside selfish agents, both in a $2$-player Markov game, and in $5$-player intertemporal social dilemmas. We analyse the resulting policies to show that the reciprocating agents are strongly influenced by their co-players' behavior.