Cooperative Heterogeneous Deep Reinforcement Learning
This work addresses the challenge of leveraging diverse agent capabilities in reinforcement learning, offering an incremental improvement for the field.
The paper tackles the problem of integrating strengths from heterogeneous deep reinforcement learning agents by proposing a Cooperative Heterogeneous Deep Reinforcement Learning (CHDRL) framework, which achieves better performance on continuous control tasks from the Mujoco benchmark compared to state-of-the-art baselines.
Numerous deep reinforcement learning agents have been proposed, and each of them has its strengths and flaws. In this work, we present a Cooperative Heterogeneous Deep Reinforcement Learning (CHDRL) framework that can learn a policy by integrating the advantages of heterogeneous agents. Specifically, we propose a cooperative learning framework that classifies heterogeneous agents into two classes: global agents and local agents. Global agents are off-policy agents that can utilize experiences from the other agents. Local agents are either on-policy agents or population-based evolutionary algorithms (EAs) agents that can explore the local area effectively. We employ global agents, which are sample-efficient, to guide the learning of local agents so that local agents can benefit from sample-efficient agents and simultaneously maintain their advantages, e.g., stability. Global agents also benefit from effective local searches. Experimental studies on a range of continuous control tasks from the Mujoco benchmark show that CHDRL achieves better performance compared with state-of-the-art baselines.