Exploring LLM-based Student Simulation for Metacognitive Cultivation
This work addresses the challenge of authentic student simulation for metacognitive education, particularly for those with learning difficulties, though it appears incremental as it builds on existing simulation methods with improved filtering and evaluation.
The paper tackled the problem of simulating students with learning difficulties using large language models to refine pedagogical methods, proposing a pipeline that automatically generates and filters high-quality simulated student agents with a two-round automated scoring system validated by human experts, resulting in efficient identification of high-quality agents and discussion of traits influencing simulation effectiveness.
Metacognitive education plays a crucial role in cultivating students' self-regulation and reflective thinking, providing essential support for those with learning difficulties through academic advising. Simulating students with insufficient learning capabilities using large language models offers a promising approach to refining pedagogical methods without ethical concerns. However, existing simulations often fail to authentically represent students' learning struggles and face challenges in evaluation due to the lack of reliable metrics and ethical constraints in data collection. To address these issues, we propose a pipeline for automatically generating and filtering high-quality simulated student agents. Our approach leverages a two-round automated scoring system validated by human experts and employs a score propagation module to obtain more consistent scores across the student graph. Experimental results demonstrate that our pipeline efficiently identifies high-quality student agents, and we discuss the traits that influence the simulation's effectiveness. By simulating students with varying degrees of learning difficulties, our work paves the way for broader applications in personalized learning and educational assessment.