LLM-Enhanced Reinforcement Learning for Long-Term User Satisfaction in Interactive Recommendation
This work addresses the challenge of long-term user satisfaction in interactive recommendation systems, which is an incremental advancement over existing methods that often neglect evolving user interests.
The paper tackles the problem of content homogeneity and filter bubbles in interactive recommender systems by proposing LLM-Enhanced Reinforcement Learning (LERL), a hierarchical framework that integrates LLM-based semantic planning with RL for fine-grained adaptation, resulting in significant improvements in long-term user satisfaction compared to state-of-the-art baselines.
Interactive recommender systems can dynamically adapt to user feedback, but often suffer from content homogeneity and filter bubble effects due to overfitting short-term user preferences. While recent efforts aim to improve content diversity, they predominantly operate in static or one-shot settings, neglecting the long-term evolution of user interests. Reinforcement learning provides a principled framework for optimizing long-term user satisfaction by modeling sequential decision-making processes. However, its application in recommendation is hindered by sparse, long-tailed user-item interactions and limited semantic planning capabilities. In this work, we propose LLM-Enhanced Reinforcement Learning (LERL), a novel hierarchical recommendation framework that integrates the semantic planning power of LLM with the fine-grained adaptability of RL. LERL consists of a high-level LLM-based planner that selects semantically diverse content categories, and a low-level RL policy that recommends personalized items within the selected semantic space. This hierarchical design narrows the action space, enhances planning efficiency, and mitigates overexposure to redundant content. Extensive experiments on real-world datasets demonstrate that LERL significantly improves long-term user satisfaction when compared with state-of-the-art baselines. The implementation of LERL is available at https://anonymous.4open.science/r/code3-18D3/.