Continual Low-Rank Adapters for LLM-based Generative Recommender Systems
This addresses the problem of adapting LLM-based recommender systems to evolving user preferences over time, which is an incremental improvement over existing continual learning methods in the domain of recommendation.
The paper tackles the challenge of continual learning in LLM-based recommender systems, where existing methods focus on preserving past preferences, which can harm performance when user interests shift. The proposed PESO method introduces a proximal regularizer to balance adaptation and preservation, and it consistently outperforms existing LoRA-based continual learning methods in empirical evaluations.
While large language models (LLMs) achieve strong performance in recommendation, they face challenges in continual learning as users, items, and user preferences evolve over time. Existing LoRA-based continual methods primarily focus on preserving performance on previous tasks, but this overlooks the unique nature of recommendation: the goal is not to predict past preferences, and outdated preferences can even harm performance when current interests shift significantly. To address this, we propose PESO (Proximally rEgularized Single evolving lOra, a continual adaptation method for LoRA in recommendation. PESO introduces a proximal regularizer that anchors the current adapter to its most recent frozen state, enabling the model to flexibly balance adaptation and preservation, and to better capture recent user behaviors. Theoretically, we show that this proximal design provides data-aware, direction-wise guidance in the LoRA subspace. Empirically, PESO consistently outperforms existing LoRA-based continual learning methods.