Emergence of Fair Leaders via Mediators in Multi-Agent Reinforcement Learning
This addresses fairness issues in multi-agent systems for applications like autonomous systems or game theory, but it is incremental as it builds on existing mediator and Stackelberg game concepts.
The paper tackles the problem of unfair leader selection in multi-agent Stackelberg games, where self-interested agents can cause biased outcomes, and proposes a mediator-based reinforcement learning framework that increases fairness in agents' returns.
Stackelberg games and their resulting equilibria have received increasing attention in the multi-agent reinforcement learning literature. Each stage of a traditional Stackelberg game involves a leader(s) acting first, followed by the followers. In situations where the roles of leader(s) and followers can be interchanged, the designated role can have considerable advantages, for example, in first-mover advantage settings. Then the question arises: Who should be the leader and when? A bias in the leader selection process can lead to unfair outcomes. This problem is aggravated if the agents are self-interested and care only about their goals and rewards. We formally define this leader selection problem and show its relation to fairness in agents' returns. Furthermore, we propose a multi-agent reinforcement learning framework that maximizes fairness by integrating mediators. Mediators have previously been used in the simultaneous action setting with varying levels of control, such as directly performing agents' actions or just recommending them. Our framework integrates mediators in the Stackelberg setting with minimal control (leader selection). We show that the presence of mediators leads to self-interested agents taking fair actions, resulting in higher overall fairness in agents' returns.