Large Language Model-Enhanced Reinforcement Learning for Diverse and Novel Recommendations
This work addresses the challenge of enhancing diversity and novelty in recommendation systems for users, though it is incremental as it builds on existing reinforcement learning and LLM integration approaches.
The paper tackled the problem of improving diversity and novelty in recommendation systems, which often prioritize click relevance, by proposing LAAC, a method that uses large language models to suggest novel items and trains a lightweight policy to refine them, resulting in outperforming existing baselines in diversity, novelty, and accuracy on real-world datasets.
In recommendation systems, diversity and novelty are essential for capturing varied user preferences and encouraging exploration, yet many systems prioritize click relevance. While reinforcement learning (RL) has been explored to improve diversity, it often depends on random exploration that may not align with user interests. We propose LAAC (LLM-guided Adversarial Actor Critic), a novel method that leverages large language models (LLMs) as reference policies to suggest novel items, while training a lightweight policy to refine these suggestions using system-specific data. The method formulates training as a bilevel optimization between actor and critic networks, enabling the critic to selectively favor promising novel actions and the actor to improve its policy beyond LLM recommendations. To mitigate overestimation of unreliable LLM suggestions, we apply regularization that anchors critic values for unexplored items close to well-estimated dataset actions. Experiments on real-world datasets show that LAAC outperforms existing baselines in diversity, novelty, and accuracy, while remaining robust on imbalanced data, effectively integrating LLM knowledge without expensive fine-tuning.