Optimizing User Profiles via Contextual Bandits for Retrieval-Augmented LLM Personalization
This work addresses the challenge of adapting LLM responses to individual users for improved personalization, representing a novel method for a known bottleneck rather than a foundational advance.
The paper tackles the problem of unreliable semantic relevance in retrieval-augmented LLM personalization by proposing PURPLE, a contextual bandit framework that optimizes user profiles, resulting in consistent outperformance over baselines across nine tasks in effectiveness and efficiency.
Large language models (LLMs) excel at general-purpose tasks, yet adapting their responses to individual users remains challenging. Retrieval augmentation provides a lightweight alternative to fine-tuning by conditioning LLMs on user history records, and existing approaches typically select these records based on semantic relevance. We argue that relevance serves as an unreliable proxy for utility: a record may be semantically similar to a query yet fail to improve generation quality or even degrade it due to redundancy or conflicting information. To bridge this gap, we propose PURPLE, a contextual bandit framework that oPtimizes UseR Profiles for LLM pErsonalization. In contrast to a greedy selection of the most relevant records, PURPLE treats profile construction as an order-sensitive generation process and utilizes a Plackett-Luce ranking model to capture complex inter-record dependencies. By training with semantically rich feedback provided by the likelihood of the reference response, our method aligns retrieval directly with generation quality. Extensive experiments on nine personalization tasks demonstrate that PURPLE consistently outperforms strong heuristic and retrieval-augmented baselines in both effectiveness and efficiency, establishing a principled and scalable solution for optimizing user profiles.