Reasoning Meets Personalization: Unleashing the Potential of Large Reasoning Model for Personalized Generation
This work tackles the problem of improving personalization in AI systems, particularly for users interacting with large language models, but it is incremental as it builds on existing reasoning models to address specific bottlenecks.
The paper systematically evaluated large reasoning models (LRMs) for personalization tasks and found they do not consistently outperform general-purpose LLMs, especially in retrieval-intensive scenarios, due to limitations like divergent thinking and misalignment. To address this, the authors proposed a novel framework called Reinforced Reasoning for Personalization (RRP) with a hierarchical reasoning template and intervention methods, which significantly outperformed existing techniques in experiments.
Personalization is a critical task in modern intelligent systems, with applications spanning diverse domains, including interactions with large language models (LLMs). Recent advances in reasoning capabilities have significantly enhanced LLMs, enabling unprecedented performance in tasks such as mathematics and coding. However, their potential for personalization tasks remains underexplored. In this paper, we present the first systematic evaluation of large reasoning models (LRMs) for personalization tasks. Surprisingly, despite generating more tokens, LRMs do not consistently outperform general-purpose LLMs, especially in retrieval-intensive scenarios where their advantages diminish. Our analysis identifies three key limitations: divergent thinking, misalignment of response formats, and ineffective use of retrieved information. To address these challenges, we propose Reinforced Reasoning for Personalization (\model), a novel framework that incorporates a hierarchical reasoning thought template to guide LRMs in generating structured outputs. Additionally, we introduce a reasoning process intervention method to enforce adherence to designed reasoning patterns, enhancing alignment. We also propose a cross-referencing mechanism to ensure consistency. Extensive experiments demonstrate that our approach significantly outperforms existing techniques.