Evaluating Pretrained models for Deployable Lifelong Learning
This work addresses the challenge of continual learning for visual RL systems, enabling deployment on unseen tasks, though it appears incremental in combining existing techniques like FSCIL with pretraining.
The authors tackled the problem of deployable lifelong learning in visual reinforcement learning by creating a novel benchmark (DeLL) and proposing a scalable system using few-shot class incremental learning, achieving scalability with small memory footprint and fewer computational resources.
We create a novel benchmark for evaluating a Deployable Lifelong Learning system for Visual Reinforcement Learning (RL) that is pretrained on a curated dataset, and propose a novel Scalable Lifelong Learning system capable of retaining knowledge from the previously learnt RL tasks. Our benchmark measures the efficacy of a deployable Lifelong Learning system that is evaluated on scalability, performance and resource utilization. Our proposed system, once pretrained on the dataset, can be deployed to perform continual learning on unseen tasks. Our proposed method consists of a Few Shot Class Incremental Learning (FSCIL) based task-mapper and an encoder/backbone trained entirely using the pretrain dataset. The policy parameters corresponding to the recognized task are then loaded to perform the task. We show that this system can be scaled to incorporate a large number of tasks due to the small memory footprint and fewer computational resources. We perform experiments on our DeLL (Deployment for Lifelong Learning) benchmark on the Atari games to determine the efficacy of the system.