Automated Knowledge Component Generation for Interpretable Knowledge Tracing in Coding Problems
This addresses the problem of reducing manual effort for educators and improving personalized learning in online coding education, though it is incremental as it builds on existing LLM and knowledge tracing methods.
The paper tackles the labor-intensive task of manually creating knowledge components (KCs) for coding problems by introducing an automated LLM-based pipeline for KC generation and tagging, and shows that using these KCs in a knowledge tracing framework outperforms existing methods and human-written KCs in predicting student responses.
Knowledge components (KCs) mapped to problems help model student learning, tracking their mastery levels on fine-grained skills thereby facilitating personalized learning and feedback in online learning platforms. However, crafting and tagging KCs to problems, traditionally performed by human domain experts, is highly labor intensive. We present an automated, LLM-based pipeline for KC generation and tagging for open-ended programming problems. We also develop an LLM-based knowledge tracing (KT) framework to leverage these LLM-generated KCs, which we refer to as KCGen-KT. We conduct extensive quantitative and qualitative evaluations on two real-world student code submission datasets in different programming languages.We find that KCGen-KT outperforms existing KT methods and human-written KCs on future student response prediction. We investigate the learning curves of generated KCs and show that LLM-generated KCs result in a better fit than human written KCs under a cognitive model. We also conduct a human evaluation with course instructors to show that our pipeline generates reasonably accurate problem-KC mappings.