Continual Gradient Low-Rank Projection Fine-Tuning for LLMs
This addresses the problem of catastrophic forgetting and limited expressiveness in continual learning for LLMs, offering an incremental improvement over existing low-rank adaptation techniques.
The paper tackles the trade-off between efficiency and expressiveness in continual fine-tuning of LLMs by proposing GORP, a training strategy that combines full and low-rank parameters in a unified gradient subspace, achieving superior performance on benchmarks compared to state-of-the-art methods.
Continual fine-tuning of Large Language Models (LLMs) is hampered by the trade-off between efficiency and expressiveness. Low-Rank Adaptation (LoRA) offers efficiency but constrains the model's ability to learn new tasks and transfer knowledge due to its low-rank nature and reliance on explicit parameter constraints. We propose GORP (Gradient LOw Rank Projection) for Continual Learning, a novel training strategy that overcomes these limitations by synergistically combining full and low-rank parameters and jointly updating within a unified low-rank gradient subspace. GORP expands the optimization space while preserving efficiency and mitigating catastrophic forgetting. Extensive experiments on continual learning benchmarks demonstrate GORP's superior performance compared to existing state-of-the-art approaches. Code is available at https://github.com/Wcxwcxw/GORP.