Evaluating Alternative Training Interventions Using Personalized Computational Models of Learning
This work addresses the challenge for instructional designers of reducing the cost and time of A/B testing by providing a simulation-based alternative, though it is incremental in applying existing modeling techniques to a specific educational domain.
The paper tackled the problem of evaluating training interventions in education by developing personalized computational models of learning to simulate student outcomes, showing that personalized models predict student behavior better than generic ones and align with previous human findings.
Evaluating different training interventions to determine which produce the best learning outcomes is one of the main challenges faced by instructional designers. Typically, these designers use A/B experiments to evaluate each intervention; however, it is costly and time consuming to run such studies. To address this issue, we explore how computational models of learning might support designers in reasoning causally about alternative interventions within a fractions tutor. We present an approach for automatically tuning models to specific individuals and show that personalized models make better predictions of students' behavior than generic ones. Next, we conduct simulations to generate counterfactual predictions of performance and learning for two students (high and low performing) in different versions of the fractions tutor. Our approach makes predictions that align with previous human findings, as well as testable predictions that might be evaluated with future human experiments.