Self-Composing Policies for Scalable Continual Reinforcement Learning
This addresses the problem of scalable continual learning in reinforcement learning for AI systems, with incremental improvements in parameter efficiency and plasticity.
The paper tackles catastrophic forgetting and interference in continual reinforcement learning by introducing a growable modular neural network architecture that selectively combines previous policies, achieving greater knowledge transfer and performance than alternative methods in benchmark continuous control and visual problems.
This work introduces a growable and modular neural network architecture that naturally avoids catastrophic forgetting and interference in continual reinforcement learning. The structure of each module allows the selective combination of previous policies along with its internal policy, accelerating the learning process on the current task. Unlike previous growing neural network approaches, we show that the number of parameters of the proposed approach grows linearly with respect to the number of tasks, and does not sacrifice plasticity to scale. Experiments conducted in benchmark continuous control and visual problems reveal that the proposed approach achieves greater knowledge transfer and performance than alternative methods.