When Do LLM Personas Support Visualization Design? A Cross-Model Study of Color Assignment and Chart Choice
For visualization designers using LLM personas as user proxies, this work reveals that persona outputs are unreliable and model-dependent, cautioning against treating them as substitutes for human participants.
The study tested whether LLM personas (GPT-4o-mini, GPT-4.1-mini, GPT-5-mini) with Big Five profiles produce stable personality effects in color assignment and chart choice tasks. Results show personality-color coupling is model-dependent (absent in GPT-4o-mini, consistent in GPT-4.1-mini, partial in GPT-5-mini), and task context drives chart choice more than personality, with no-persona baselines matching top choices in 8 of 9 cases.
Large language model personas are increasingly used to approximate diverse users during early-stage visualization design, but it remains unclear whether persona-conditioned outputs reflect stable personality effects or artifacts of model choice and task framing. We examine this question across two visualization-relevant tasks: color assignment for abstract and concrete concepts, and chart-idiom preference ratings across task contexts. Using 43 Big Five profiles across GPT-4o-mini, GPT-4.1-mini, and GPT-5-mini, we find that personality-color coupling is highly model-configuration dependent: absent in GPT-4o-mini for all six concepts, consistent in GPT-4.1-mini across all six, and partial in GPT-5-mini for two of six. Concept type further shapes the signal: for abstract concepts, personality explains more hue variance than model identity, while concrete concepts show smaller and comparable effects. In chart choice, trait-aligned cluster aggregation produces stable top-idiom rankings across all nine cluster-context combinations, but a no-persona baseline recovers the same top choice in 8 of 9 model-context cells, indicating that task context drives rank-1 selection more than personality. These findings position LLM personas as exploratory probes for visualization design, not substitutes for human participants, and motivate multi-model testing, concept-type disaggregation, and no-persona baselines in future studies.