What LLMs Miss in Recommendations: Bridging the Gap with Retrieval-Augmented Collaborative Signals
This addresses the gap in LLM-based recommender systems for users needing better collaborative filtering, though it is incremental by enhancing existing methods.
The paper tackled the problem of whether large language models (LLMs) can effectively use collaborative signals from user-item interactions for recommendations, finding that current LLMs often fall short but a retrieval-augmented generation (RAG) method substantially improves recommendation quality.
User-item interactions contain rich collaborative signals that form the backbone of many successful recommender systems. While recent work has explored the use of large language models (LLMs) for recommendation, it remains unclear whether LLMs can effectively reason over this type of collaborative information. In this paper, we conduct a systematic comparison between LLMs and classical matrix factorization (MF) models to assess LLMs' ability to leverage user-item interaction data. We further introduce a simple retrieval-augmented generation (RAG) method that enhances LLMs by grounding their predictions in structured interaction data. Our experiments reveal that current LLMs often fall short in capturing collaborative patterns inherent to MF models, but that our RAG-based approach substantially improves recommendation quality-highlighting a promising direction for future LLM-based recommenders.