Using and Designing Platforms for In Vivo Education Experiments
This work addresses the challenge of enhancing educational tools through in vivo experiments, but it is incremental as it builds on existing A/B testing infrastructure.
The paper tackles the problem of integrating randomized experiments into online educational platforms to drive practical improvements, demonstrating examples on Khan Academy, edX, and ASSISTments.
In contrast to typical laboratory experiments, the everyday use of online educational resources by large populations and the prevalence of software infrastructure for A/B testing leads us to consider how platforms can embed in vivo experiments that do not merely support research, but ensure practical improvements to their educational components. Examples are presented of randomized experimental comparisons conducted by subsets of the authors in three widely used online educational platforms Khan Academy, edX, and ASSISTments. We suggest design principles for platform technology to support randomized experiments that lead to practical improvements enabling Iterative Improvement and Collaborative Work and explain the benefit of their implementation by WPI co-authors in the ASSISTments platform.