Exploring the Benefits of Teams in Multiagent Learning
This addresses the challenge of cooperation in multiagent learning, particularly in social dilemmas, though it appears incremental by adapting existing psychological concepts to reinforcement learning.
The paper tackles the problem of fostering cooperation in multiagent systems by proposing a new model of multiagent teams inspired by organizational psychology, and finds that agents in teams develop cooperative policies, coordinate better, and achieve higher rewards in social dilemmas.
For problems requiring cooperation, many multiagent systems implement solutions among either individual agents or across an entire population towards a common goal. Multiagent teams are primarily studied when in conflict; however, organizational psychology (OP) highlights the benefits of teams among human populations for learning how to coordinate and cooperate. In this paper, we propose a new model of multiagent teams for reinforcement learning (RL) agents inspired by OP and early work on teams in artificial intelligence. We validate our model using complex social dilemmas that are popular in recent multiagent RL and find that agents divided into teams develop cooperative pro-social policies despite incentives to not cooperate. Furthermore, agents are better able to coordinate and learn emergent roles within their teams and achieve higher rewards compared to when the interests of all agents are aligned.