Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs
This addresses the challenge of personalizing conversational AI for users, but it is incremental as it focuses on benchmarking and improving existing methods rather than proposing a new paradigm.
The authors tackled the problem of LLMs' limited ability to personalize responses to user preferences by introducing PrefEval, a benchmark for evaluating preference following in long-context conversations, and found that state-of-the-art LLMs perform poorly, with accuracy below 10% in zero-shot settings at 10 turns, though fine-tuning on PrefEval significantly improves performance.
Large Language Models (LLMs) are increasingly used as chatbots, yet their ability to personalize responses to user preferences remains limited. We introduce PrefEval, a benchmark for evaluating LLMs' ability to infer, memorize and adhere to user preferences in a long-context conversational setting. PrefEval comprises 3,000 manually curated user preference and query pairs spanning 20 topics. PrefEval contains user personalization or preference information in both explicit and implicit forms, and evaluates LLM performance using a generation and a classification task. With PrefEval, we evaluated the aforementioned preference following capabilities of 10 open-source and proprietary LLMs in multi-session conversations with varying context lengths up to 100k tokens. We benchmark with various prompting, iterative feedback, and retrieval-augmented generation methods. Our benchmarking effort reveals that state-of-the-art LLMs face significant challenges in proactively following users' preferences during conversations. In particular, in zero-shot settings, preference following accuracy falls below 10% at merely 10 turns (~3k tokens) across most evaluated models. Even with advanced prompting and retrieval methods, preference following still deteriorates in long-context conversations. Furthermore, we show that fine-tuning on PrefEval significantly improves performance. We believe PrefEval serves as a valuable resource for measuring, understanding, and enhancing LLMs' preference following abilities, paving the way for personalized conversational agents. Our code and dataset are available at https://prefeval.github.io/.