Next Token Knowledge Tracing: Exploiting Pretrained LLM Representations to Decode Student Behaviour
This work addresses the challenge of personalized learning in AI education by enhancing predictive accuracy for student behavior, though it is incremental in leveraging existing LLM methods.
The paper tackled the problem of predicting student responses in educational settings by incorporating question text into knowledge tracing, achieving significant performance improvements over state-of-the-art neural models and better generalization to cold-start scenarios.
Modelling student knowledge is a key challenge when leveraging AI in education, with major implications for personalised learning. The Knowledge Tracing (KT) task aims to predict how students will respond to educational questions in learning environments, based on their prior interactions. Existing KT models typically use response correctness along with metadata like skill tags and timestamps, often overlooking the question text, which is an important source of pedagogical insight. This omission poses a lost opportunity while limiting predictive performance. We propose Next Token Knowledge Tracing (NTKT), a novel approach that reframes KT as a next-token prediction task using pretrained Large Language Models (LLMs). NTKT represents both student histories and question content as sequences of text, allowing LLMs to learn patterns in both behaviour and language. Our series of experiments significantly improves performance over state-of-the-art neural KT models and generalises much better to cold-start questions and users. These findings highlight the importance of question content in KT and demonstrate the benefits of leveraging pretrained representations of LLMs to model student learning more effectively.