Multi-Agent Actor-Critic with Generative Cooperative Policy Network
This work addresses the challenge of decentralized policy optimization in multi-agent systems, which is incremental as it builds on existing methods by introducing a generative cooperative network to enhance training.
The paper tackles the problem of deriving equilibrium strategies for collaborative multi-agent tasks in Markov games by proposing a multi-agent reinforcement learning approach with two policy networks, resulting in more effective policy exploration and improved performance for collaborative tasks.
We propose an efficient multi-agent reinforcement learning approach to derive equilibrium strategies for multi-agents who are participating in a Markov game. Mainly, we are focused on obtaining decentralized policies for agents to maximize the performance of a collaborative task by all the agents, which is similar to solving a decentralized Markov decision process. We propose to use two different policy networks: (1) decentralized greedy policy network used to generate greedy action during training and execution period and (2) generative cooperative policy network (GCPN) used to generate action samples to make other agents improve their objectives during training period. We show that the samples generated by GCPN enable other agents to explore the policy space more effectively and favorably to reach a better policy in terms of achieving the collaborative tasks.