Learning Coordinated Preference for Multi-Objective Multi-Agent Reinforcement Learning
For researchers in multi-agent reinforcement learning, this work introduces a method to handle conflicting objectives across agents, improving coordination in cooperative settings.
This paper addresses cooperative multi-objective multi-agent reinforcement learning (MOMARL) by proposing Preference Coordinated Multi-agent Policy Optimization (PCMA), which learns coordinated agent-specific preferences to enable complementary trade-offs among agents. Experiments show PCMA improves both performance and trade-off coordination in multiple environments and a traffic-control scenario.
Cooperative multi-objective multi-agent reinforcement learning (MOMARL) models team decision making under multiple, potentially conflicting objectives. In this setting, conflicts arise not only across objectives but also across agents with different observations, roles, and contributions. We propose Preference Coordinated Multi-agent Policy Optimization (PCMA), which learns coordinated agent-specific preferences to enable complementary trade-offs among agents. Theoretically, we formulate cooperative MOMARL as a team-optimal game and show that, under suitable conditions, preference diversity can induce team improvement through a first-order improvement decomposition. Experiments on multiple cooperative MOMA environments and a practical traffic-control scenario show that PCMA improves both performance and trade-off coordination.