Learning Personalized Alignment for Evaluating Open-ended Text Generation
This addresses the need for personalized evaluation metrics in AI content generation, offering an incremental improvement over existing methods.
The paper tackles the problem of evaluating large language models' alignment with diverse human preferences in open-ended text generation, introducing PerSE, which shows a 15.8% increase in Kendall correlation and 13.7% rise in accuracy compared to GPT-4.
Recent research has increasingly focused on evaluating large language models' (LLMs) alignment with diverse human values and preferences, particularly for open-ended tasks like story generation. Traditional evaluation metrics rely heavily on lexical similarity with human-written references, often showing poor correlation with human judgments and failing to account for alignment with the diversity of human preferences. To address these challenges, we introduce PerSE, an interpretable evaluation framework designed to assess alignment with specific human preferences. It is tuned to infer specific preferences from an in-context personal profile and evaluate the alignment between the generated content and personal preferences. PerSE enhances interpretability by providing detailed comments and fine-grained scoring, facilitating more personalized content generation. Our 13B LLaMA-2-based PerSE shows a 15.8% increase in Kendall correlation and a 13.7% rise in accuracy with zero-shot reviewers compared to GPT-4. It also outperforms GPT-4 by 46.01% in Kendall correlation on new domains, indicating its transferability.