The Impact of Steering Large Language Models with Persona Vectors in Educational Applications
This work addresses the problem of unpredictable model behavior in educational applications, highlighting risks for students and educators, but it is incremental as it builds on existing activation-based steering methods.
The study investigated how steering large language models with persona vectors affects answer generation and automated scoring in educational tasks, finding that persona steering generally lowers answer quality, with up to 11x greater sensitivity in open-ended tasks, and causes predictable calibration shifts in scoring, such as harsher grades from 'evil' personas.
Activation-based steering can personalize large language models at inference time, but its effects in educational settings remain unclear. We study persona vectors for seven character traits in short-answer generation and automated scoring on the ASAP-SAS benchmark across three models spanning two architectures. Persona steering lowers answer quality overall, with much larger effects on open-ended English Language Arts (ELA) prompts than on factual science prompts; interpretive and argumentative tasks are up to 11x more sensitive. On the scoring side, we observe predictable valence-aligned calibration shifts: evil and impolite scorers grade more harshly, while good and optimistic scorers grade more leniently. ELA tasks are 2.5-3x more susceptible to scorer personalization than science tasks, and the Mixture-of-Experts model shows roughly 6x larger calibration shifts than the dense models. To our knowledge, this is the first study to systematically examine the effects of activation-steered persona traits in educational generation and scoring, and the results highlight the need for task-aware and architecture-aware calibration when deploying steered models in educational settings.