KARL: Knowledge-Aware Retrieval and Representations aid Retention and Learning in Students
This addresses the problem of inefficient learning in students using flashcard systems, though it is incremental as it builds on existing methods like deep knowledge tracing.
The paper tackled the problem of flashcard schedulers ignoring card content by proposing content-aware scheduling, and KARL, a model using deep knowledge tracing, retrieval, and BERT, improved learning efficiency over state-of-the-art methods based on a dataset of 123,143 study logs and 32 study paths from 27 users.
Flashcard schedulers rely on 1) student models to predict the flashcards a student knows; and 2) teaching policies to pick which cards to show next via these predictions. Prior student models, however, just use study data like the student's past responses, ignoring the text on cards. We propose content-aware scheduling, the first schedulers exploiting flashcard content. To give the first evidence that such schedulers enhance student learning, we build KARL, a simple but effective content-aware student model employing deep knowledge tracing (DKT), retrieval, and BERT to predict student recall. We train KARL by collecting a new dataset of 123,143 study logs on diverse trivia questions. KARL bests existing student models in AUC and calibration error. To ensure our improved predictions lead to better student learning, we create a novel delta-based teaching policy to deploy KARL online. Based on 32 study paths from 27 users, KARL improves learning efficiency over SOTA, showing KARL's strength and encouraging researchers to look beyond historical study data to fully capture student abilities.