No Task Left Behind: Multi-Task Learning of Knowledge Tracing and Option Tracing for Better Student Assessment
This work addresses the problem of more precise student assessment in AI Education, particularly for multiple-choice questions, by integrating binary correctness and option choice information, representing an incremental improvement over existing methods.
The paper tackles the limitation of conventional Knowledge Tracing (KT) and Option Tracing (OT) in student assessment by proposing a multi-task learning framework called DP-MTL, which combines KT and OT to improve both tasks and downstream score prediction, with experimental confirmation of significant performance gains.
Student assessment is one of the most fundamental tasks in the field of AI Education (AIEd). One of the most common approach to student assessment is Knowledge Tracing (KT), which evaluates a student's knowledge state by predicting whether the student will answer a given question correctly or not. However, in the context of multiple choice (polytomous) questions, conventional KT approaches are limited in that they only consider the binary (dichotomous) correctness label (i.e., correct or incorrect), and disregard the specific option chosen by the student. Meanwhile, Option Tracing (OT) attempts to model a student by predicting which option they will choose for a given question, but overlooks the correctness information. In this paper, we propose Dichotomous-Polytomous Multi-Task Learning (DP-MTL), a multi-task learning framework that combines KT and OT for more precise student assessment. In particular, we show that the KT objective acts as a regularization term for OT in the DP-MTL framework, and propose an appropriate architecture for applying our method on top of existing deep learning-based KT models. We experimentally confirm that DP-MTL significantly improves both KT and OT performances, and also benefits downstream tasks such as Score Prediction (SP).