DisCoRL: Continual Reinforcement Learning via Policy Distillation
This work addresses the problem of continual learning in reinforcement learning for robotics, though it appears incremental as it builds on existing techniques like policy distillation.
The paper tackled the challenges of multi-task and continual reinforcement learning by proposing DisCoRL, which combines state representation learning and policy distillation, enabling a single model to learn and infer policies across sequential tasks without forgetting, and demonstrated its effectiveness in simulated and real-life settings.
In multi-task reinforcement learning there are two main challenges: at training time, the ability to learn different policies with a single model; at test time, inferring which of those policies applying without an external signal. In the case of continual reinforcement learning a third challenge arises: learning tasks sequentially without forgetting the previous ones. In this paper, we tackle these challenges by proposing DisCoRL, an approach combining state representation learning and policy distillation. We experiment on a sequence of three simulated 2D navigation tasks with a 3 wheel omni-directional robot. Moreover, we tested our approach's robustness by transferring the final policy into a real life setting. The policy can solve all tasks and automatically infer which one to run.