Lifelong Language-Conditioned Robotic Manipulation Learning
This work tackles the problem of catastrophic forgetting in robotic manipulation for dynamic scene deployment, which is an incremental improvement for roboticists.
This paper addresses catastrophic forgetting in language-conditioned robotic manipulation agents when sequentially adapting to new skills. The authors propose SkillsCrafter, a framework that continually learns multiple skills while reducing forgetting and facilitating the learning of new skills by retaining old skill knowledge and inheriting shared knowledge.
Traditional language-conditioned manipulation agent sequential adaptation to new manipulation skills leads to catastrophic forgetting of old skills, limiting dynamic scene practical deployment. In this paper, we propose SkillsCrafter, a novel robotic manipulation framework designed to continually learn multiple skills while reducing catastrophic forgetting of old skills. Specifically, we propose a Manipulation Skills Adaptation to retain the old skills knowledge while inheriting the shared knowledge between new and old skills to facilitate learning of new skills. Meanwhile, we perform the singular value decomposition on the diverse skill instructions to obtain common skill semantic subspace projection matrices, thereby recording the essential semantic space of skills. To achieve forget-less and generalization manipulation, we propose a Skills Specialization Aggregation to compute inter-skills similarity in skill semantic subspaces, achieving aggregation of the previously learned skill knowledge for any new or unknown skill. Extensive experiments demonstrate the effectiveness and superiority of our proposed SkillsCrafter.