Stable and Explainable Personality Trait Evaluation in Large Language Models with Internal Activations
This addresses the need for reliable model interpretation and responsible deployment in AI, though it is incremental as it builds on existing evaluation methods.
The paper tackled the problem of unstable and unexplainable personality trait evaluation in Large Language Models by proposing Persona-Vector Neutrality Interpolation (PVNI), which uses internal activations to achieve more stable evaluations, as demonstrated in experiments across diverse LLMs.
Evaluating personality traits in Large Language Models (LLMs) is key to model interpretation, comparison, and responsible deployment. However, existing questionnaire-based evaluation methods exhibit limited stability and offer little explainability, as their results are highly sensitive to minor variations in prompt phrasing or role-play configurations. To address these limitations, we propose an internal-activation-based approach, termed Persona-Vector Neutrality Interpolation (PVNI), for stable and explainable personality trait evaluation in LLMs. PVNI extracts a persona vector associated with a target personality trait from the model's internal activations using contrastive prompts. It then estimates the corresponding neutral score by interpolating along the persona vector as an anchor axis, enabling an interpretable comparison between the neutral prompt representation and the persona direction. We provide a theoretical analysis of the effectiveness and generalization properties of PVNI. Extensive experiments across diverse LLMs demonstrate that PVNI yields substantially more stable personality trait evaluations than existing methods, even under questionnaire and role-play variants.