LLMInit: A Free Lunch from Large Language Models for Selective Initialization of Recommendation
This work provides a scalable solution for industrial recommender systems to leverage LLMs without high computational overhead, though it is incremental in adapting existing methods to a new context.
The paper tackles the challenge of integrating large language models (LLMs) into recommender systems to address cold-start and data-sparse issues, proposing LLMInit, a framework that uses selective initialization strategies to improve recommendation performance with low computational costs, as demonstrated by significant gains in experiments on real-world datasets.
Collaborative filtering (CF) is widely adopted in industrial recommender systems (RecSys) for modeling user-item interactions across numerous applications, but often struggles with cold-start and data-sparse scenarios. Recent advancements in pre-trained large language models (LLMs) with rich semantic knowledge, offer promising solutions to these challenges. However, deploying LLMs at scale is hindered by their significant computational demands and latency. In this paper, we propose a novel and scalable LLM-RecSys framework, LLMInit, designed to integrate pretrained LLM embeddings into CF models through selective initialization strategies. Specifically, we identify the embedding collapse issue observed when CF models scale and match the large embedding sizes in LLMs and avoid the problem by introducing efficient sampling methods, including, random, uniform, and variance-based selections. Comprehensive experiments conducted on multiple real-world datasets demonstrate that LLMInit significantly improves recommendation performance while maintaining low computational costs, offering a practical and scalable solution for industrial applications. To facilitate industry adoption and promote future research, we provide open-source access to our implementation at https://github.com/DavidZWZ/LLMInit.