Centralized Reward Agent for Knowledge Sharing and Transfer in Multi-Task Reinforcement Learning
This addresses the problem of inefficient learning in sparse-reward multi-task RL for AI/robotics applications, representing an incremental improvement over existing reward shaping methods.
The authors tackled the sparse-reward challenge in multi-task reinforcement learning by proposing a centralized reward agent framework that distills and shares knowledge across tasks, demonstrating improved learning efficiency and effective transfer to unseen tasks on benchmarks like Meta-World.
Reward shaping is effective in addressing the sparse-reward challenge in reinforcement learning (RL) by providing immediate feedback through auxiliary, informative rewards. Based on the reward shaping strategy, we propose a novel multi-task reinforcement learning framework that integrates a centralized reward agent (CRA) and multiple distributed policy agents. The CRA functions as a knowledge pool, aimed at distilling knowledge from various tasks and distributing it to individual policy agents to improve learning efficiency. Specifically, the shaped rewards serve as a straightforward metric for encoding knowledge. This framework not only enhances knowledge sharing across established tasks but also adapts to new tasks by transferring meaningful reward signals. We validate the proposed method on both discrete and continuous domains, including the representative Meta-World benchmark, demonstrating its robustness in multi-task sparse-reward settings and its effective transferability to unseen tasks.