Personality Matters: User Traits Predict LLM Preferences in Multi-Turn Collaborative Tasks
This addresses the problem of optimizing LLM selection for diverse users in collaborative workflows, though it is incremental as it builds on existing personality and preference research.
The study investigated whether user personality traits predict preferences for different LLMs in multi-turn collaborative tasks, finding that Rationals strongly preferred GPT-4 for goal-oriented tasks and Idealists favored Claude 3.5 for creative and analytical tasks, with other types showing task-dependent preferences.
As Large Language Models (LLMs) increasingly integrate into everyday workflows, where users shape outcomes through multi-turn collaboration, a critical question emerges: do users with different personality traits systematically prefer certain LLMs over others? We conducted a study with 32 participants evenly distributed across four Keirsey personality types, evaluating their interactions with GPT-4 and Claude 3.5 across four collaborative tasks: data analysis, creative writing, information retrieval, and writing assistance. Results revealed significant personality-driven preferences: Rationals strongly preferred GPT-4, particularly for goal-oriented tasks, while idealists favored Claude 3.5, especially for creative and analytical tasks. Other personality types showed task-dependent preferences. Sentiment analysis of qualitative feedback confirmed these patterns. Notably, aggregate helpfulness ratings were similar across models, showing how personality-based analysis reveals LLM differences that traditional evaluations miss.