Renaissance Robot: Optimal Transport Policy Fusion for Learning Diverse Skills
This addresses the challenge of reusing learned policies for new robotics tasks, offering an incremental improvement over retraining or starting from scratch.
The paper tackles the problem of time-consuming deep reinforcement learning in robotics by proposing a post-hoc policy fusion technique using Optimal Transport theory to consolidate knowledge from multiple agents trained on distinct scenarios, resulting in quicker learning of new skills with less time and computational resources.
Deep reinforcement learning (RL) is a promising approach to solving complex robotics problems. However, the process of learning through trial-and-error interactions is often highly time-consuming, despite recent advancements in RL algorithms. Additionally, the success of RL is critically dependent on how well the reward-shaping function suits the task, which is also time-consuming to design. As agents trained on a variety of robotics problems continue to proliferate, the ability to reuse their valuable learning for new domains becomes increasingly significant. In this paper, we propose a post-hoc technique for policy fusion using Optimal Transport theory as a robust means of consolidating the knowledge of multiple agents that have been trained on distinct scenarios. We further demonstrate that this provides an improved weights initialisation of the neural network policy for learning new tasks, requiring less time and computational resources than either retraining the parent policies or training a new policy from scratch. Ultimately, our results on diverse agents commonly used in deep RL show that specialised knowledge can be unified into a "Renaissance agent", allowing for quicker learning of new skills.