Analytic Subspace Routing: How Recursive Least Squares Works in Continual Learning of Large Language Model
This addresses the challenge of maintaining general skills and accumulated knowledge in LLMs during continual finetuning, offering a solution to interference between tasks without replaying data, though it appears incremental as it builds on low-rank adaptation and routing mechanisms.
The paper tackles the problem of catastrophic forgetting in continual learning for large language models by proposing Analytic Subspace Routing, which isolates task learning in subspaces and uses a router trained with Recursive Least Squares, achieving near-perfect retention of prior knowledge and seamless integration of new information.
Large Language Models (LLMs) possess encompassing capabilities that can process diverse language-related tasks. However, finetuning on LLMs will diminish this general skills and continual finetuning will further cause severe degradation on accumulated knowledge. Recently, Continual Learning (CL) in Large Language Models (LLMs) arises which aims to continually adapt the LLMs to new tasks while maintaining previously learned knowledge and inheriting general skills. Existing techniques either leverage previous data to replay, leading to extra computational costs, or utilize a single parameter-efficient module to learn the downstream task, constraining new knowledge absorption with interference between different tasks. Toward these issues, this paper proposes Analytic Subspace Routing(ASR) to address these challenges. For each task, we isolate the learning within a subspace of deep layers' features via low-rank adaptation, eliminating knowledge interference between different tasks. Additionally, we propose an analytic routing mechanism to properly utilize knowledge learned in different subspaces. Our approach employs Recursive Least Squares to train a multi-task router model, allowing the router to dynamically adapt to incoming data without requiring access to historical data. Also, the router effectively assigns the current task to an appropriate subspace and has a non-forgetting property of previously learned tasks with a solid theoretical guarantee. Experimental results demonstrate that our method achieves near-perfect retention of prior knowledge while seamlessly integrating new information, effectively overcoming the core limitations of existing methods. Our code will be released after acceptance.