KASER: Knowledge-Aligned Student Error Simulator for Open-Ended Coding Tasks
For educators and AI researchers, this work addresses the challenge of generating diverse and realistic student error simulations in coding education, which is important for building better tutoring systems.
KASER introduces a reinforcement learning-based method to simulate student errors in open-ended coding tasks, achieving better code and error prediction accuracy and diversity than baselines on two real-world datasets.
Open-ended tasks, such as coding problems that are common in computer science education, provide detailed insights into student knowledge. However, training large language models (LLMs) to simulate and predict possible student errors in their responses to these problems can be challenging: they often suffer from mode collapse and fail to fully capture the diversity in syntax, style, and solution approach in student responses. In this work, we present KASER (Knowledge-Aligned Student Error Simulator), a novel approach that aligns errors with student knowledge. We propose a training method based on reinforcement learning using a hybrid reward that reflects three aspects of student code prediction: i) code similarity to the ground-truth, ii) error matching, and iii) code prediction diversity. On two real-world datasets, we perform two levels of evaluation and show that: At the per-student-problem pair level, our method outperforms baselines on code and error prediction; at the per-problem level, our method outperforms baselines on error coverage and simulated code diversity.