Tracing Prompt-Level Trajectories to Understand Student Learning with AI in Programming Education
For educators and designers of AI tools in programming education, this work provides empirical evidence linking interaction patterns with learning outcomes, offering design implications for personalized student-AI collaboration.
This study analyzed student-LLM chat histories and code submissions from 163 students in an introductory Python assignment, identifying prompting trajectories from full delegation to iterative refinement. Most students directly copied AI-generated code, but many scaffolded code generation through iterative refinement, with trajectories serving as windows into self-regulation and learning orientation.
As AI tools such as ChatGPT enter programming classrooms, students encounter differing rules across courses and instructors, which shape how they use AI and leave them with unequal capabilities for leveraging it. We investigate how students engaged with AI in an introductory Python assignment, analyzing student-LLM chat histories and final code submissions from 163 students. We examined prompt-level strategies, traced trajectories of interaction, and compared AI-generated code with student submissions. We identified trajectories ranging from full delegation to iterative refinement, with hybrid forms in between. Although most students directly copied AI-generated code in their submission, many students scaffolded the code generation through iterative refinement. We also contrasted interaction patterns with assignment outcomes and course performance. Our findings show that prompting trajectories serve as promising windows into students' self-regulation and learning orientation. We draw design implications for educational AI systems that promote personalized and productive student-AI collaborative learning.