EpiPersona: Persona Projection and Episode Coupling for Pluralistic Preference Modeling
This addresses pluralistic alignment for adapting LLMs to diverse individual and minority group preferences, representing an incremental improvement over existing methods.
The paper tackles the problem of modeling diverse user preferences in large language models by separating stable personal traits from episode-specific factors, introducing the EpiPersona framework which achieves notable performance gains in hard episodic-shift scenarios and remains effective with sparse data.
Pluralistic alignment is essential for adapting large language models (LLMs) to the diverse preferences of individuals and minority groups. However, existing approaches often mix stable personal traits with episode-specific factors, limiting their ability to generalize across episodes. To address this challenge, we introduce EpiPersona, a framework for explicit persona-episode coupling. EpiPersona first projects noisy preference feedback into a low-dimensional persona space, where similar personas are aggregated into shared discrete codes. This process separates enduring personal characteristics from situational signals without relying on predefined preference dimensions. The inferred persona representation is then coupled with the current episode, enabling episode-aware preference prediction. Extensive experiments show that EpiPersona consistently outperforms the baselines. It achieves notable performance gains in hard episodic-shift scenarios, while remaining effective with sparse preference data.