Integrating Summarization and Retrieval for Enhanced Personalization via Large Language Models
This work improves personalization for NLP systems like voice assistants by enabling better performance under practical constraints, though it is incremental as it builds on existing retrieval-augmented methods.
The paper tackles the problem of personalizing large language models (LLMs) by addressing input length and latency issues in retrieval-based methods, proposing a summary-augmented approach that uses LLM-generated user summaries. The result shows that with 75% less retrieved user data, the method matches or outperforms retrieval augmentation on most tasks in the LaMP benchmark.
Personalization, the ability to tailor a system to individual users, is an essential factor in user experience with natural language processing (NLP) systems. With the emergence of Large Language Models (LLMs), a key question is how to leverage these models to better personalize user experiences. To personalize a language model's output, a straightforward approach is to incorporate past user data into the language model prompt, but this approach can result in lengthy inputs exceeding limitations on input length and incurring latency and cost issues. Existing approaches tackle such challenges by selectively extracting relevant user data (i.e. selective retrieval) to construct a prompt for downstream tasks. However, retrieval-based methods are limited by potential information loss, lack of more profound user understanding, and cold-start challenges. To overcome these limitations, we propose a novel summary-augmented approach by extending retrieval-augmented personalization with task-aware user summaries generated by LLMs. The summaries can be generated and stored offline, enabling real-world systems with runtime constraints like voice assistants to leverage the power of LLMs. Experiments show our method with 75% less of retrieved user data is on-par or outperforms retrieval augmentation on most tasks in the LaMP personalization benchmark. We demonstrate that offline summarization via LLMs and runtime retrieval enables better performance for personalization on a range of tasks under practical constraints.