A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models
This is an incremental survey paper that organizes existing research for researchers in NLP and personalization.
This survey analyzes personalized preference alignment techniques for large language models, categorizing them into training time, inference time, and user-modeling methods, while discussing their strengths, limitations, evaluation, benchmarks, and open problems.
Personalized preference alignment for large language models (LLMs), the process of tailoring LLMs to individual users' preferences, is an emerging research direction spanning the area of NLP and personalization. In this survey, we present an analysis of works on personalized alignment and modeling for LLMs. We introduce a taxonomy of preference alignment techniques, including training time, inference time, and additionally, user-modeling based methods. We provide analysis and discussion on the strengths and limitations of each group of techniques and then cover evaluation, benchmarks, as well as open problems in the field.