On the Opportunities of Large Language Models for Programming Process Data
This work addresses the challenge of providing automated, formative feedback on programming processes for computing education, though it appears incremental as it builds on existing LLM capabilities.
The paper explores using large language models (LLMs) to analyze programming process data, aiming to automate formative feedback for students. It includes a case study demonstrating LLMs' ability to summarize programming processes and generate feedback, moving the community closer to automated systems.
Computing educators and researchers have used programming process data to understand how programs are constructed and what sorts of problems students struggle with. Although such data shows promise for using it for feedback, fully automated programming process feedback systems have still been an under-explored area. The recent emergence of large language models (LLMs) have yielded additional opportunities for researchers in a wide variety of fields. LLMs are efficient at transforming content from one format to another, leveraging the body of knowledge they have been trained with in the process. In this article, we discuss opportunities of using LLMs for analyzing programming process data. To complement our discussion, we outline a case study where we have leveraged LLMs for automatically summarizing the programming process and for creating formative feedback on the programming process. Overall, our discussion and findings highlight that the computing education research and practice community is again one step closer to automating formative programming process-focused feedback.