Programming Knowledge Tracing: A Comprehensive Dataset and A New Model
This work addresses the need for accurate student behavior prediction in programming education, though it is incremental as it builds on existing knowledge tracing methods with domain-specific enhancements.
The paper tackles the problem of knowledge tracing in programming education by introducing a comprehensive dataset (BePKT) and a new model (PDKT) that achieves state-of-the-art performance on this dataset, with improvements verified through experiments.
In this paper, we study knowledge tracing in the domain of programming education and make two important contributions. First, we harvest and publish so far the most comprehensive dataset, namely BePKT, which covers various online behaviors in an OJ system, including programming text problems, knowledge annotations, user-submitted code and system-logged events. Second, we propose a new model PDKT to exploit the enriched context for accurate student behavior prediction. More specifically, we construct a bipartite graph for programming problem embedding, and design an improved pre-training model PLCodeBERT for code embedding, as well as a double-sequence RNN model with exponential decay attention for effective feature fusion. Experimental results on the new dataset BePKT show that our proposed model establishes state-of-the-art performance in programming knowledge tracing. In addition, we verify that our code embedding strategy based on PLCodeBERT is complementary to existing knowledge tracing models to further enhance their accuracy. As a side product, PLCodeBERT also results in better performance in other programming-related tasks such as code clone detection.