LLM-Based Educational Simulation: Evaluating Temporal Student Persona Stability Across ADHD Profiles
This work addresses the prerequisite of persona stability for valid LLM-based educational simulations, which is critical for teacher training and adaptive tutoring applications.
The study tests whether LLM-based student simulations maintain stable personas across extended interactions, using a dual-assessment framework with four ADHD persona conditions, five LLMs, and three prompt designs. Results show self-reported characteristics remain stable for high intensities, but behavioral expression drifts in unscripted dialog for high and moderate ADHD personas, which is eliminated by scripted interactions with explicit task prompts.
Student simulation with Large language models (LLMs) offers a scalable alternative for educational research and teacher training. Yet, its validity depends on whether models maintain stable personas across extended interactions. We test this prerequisite using a dual-assessment framework measuring self-reported characteristics and observer-rated behavioral expressions. Across two experiments testing four clinically-grounded ADHD persona conditions, five LLMs, and three prompt designs, we quantify between-conversation stability (N=4,968) and within-conversation stability (N=3,952 across 9 turns). Self-reported characteristics remain stable for high intensities, constituting a necessary prerequisite for valid behavioral simulation. Observer-rated behavioral expression reveals selective instability: within-conversation drift occurs in unscripted dialog for high and moderate ADHD personas. Scripted interactions with explicit task prompts eliminate this drift entirely. Stable, persona-aligned simulated learners benefit from a structured interaction design to maintain behavioral coherence, which holds significant implications for teacher training, adaptive tutoring, and any application requiring sustained, path-dependent learner interactions.