T2S: Tokenized Skill Scaling for Lifelong Imitation Learning
This work addresses the problem of lifelong skill acquisition for imitation learning systems, offering a unified framework that is incremental in improving scalability and transfer efficiency.
The paper tackles the challenge of balancing catastrophic forgetting and capacity for new skills in lifelong imitation learning by proposing Tokenized Skill Scaling (T2S), which achieves an average NBT of 1.0% to prevent forgetting, uses only 8.0% trainable tokens for scaling, and attains an average FWT of 77.7% for knowledge transfer.
The main challenge in lifelong imitation learning lies in the balance between mitigating catastrophic forgetting of previous skills while maintaining sufficient capacity for acquiring new ones. However, current approaches typically address these aspects in isolation, overlooking their internal correlation in lifelong skill acquisition. We address this limitation with a unified framework named Tokenized Skill Scaling (T2S). Specifically, by tokenizing the model parameters, the linear parameter mapping of the traditional transformer is transformed into cross-attention between input and learnable tokens, thereby enhancing model scalability through the easy extension of new tokens. Additionally, we introduce language-guided skill scaling to transfer knowledge across tasks efficiently and avoid linearly growing parameters. Extensive experiments across diverse tasks demonstrate that T2S: 1) effectively prevents catastrophic forgetting (achieving an average NBT of 1.0% across the three LIBERO task suites), 2) excels in new skill scaling with minimal increases in trainable parameters (needing only 8.0% trainable tokens in an average of lifelong tasks), and 3) enables efficient knowledge transfer between tasks (achieving an average FWT of 77.7% across the three LIBERO task suites), offering a promising solution for lifelong imitation learning.