Using Adaptive Experiments to Rapidly Help Students
This work addresses the challenge of optimizing instructional interventions for students in digital educational environments, but it is incremental as it presents a case study rather than a broad advancement.
The authors tackled the problem of improving student outcomes in educational interventions by conducting an adaptive experiment using Thompson Sampling for homework email reminders, comparing it to a traditional uniform random approach, and found it increased the chance of better outcomes, though specific numerical results were not provided.
Adaptive experiments can increase the chance that current students obtain better outcomes from a field experiment of an instructional intervention. In such experiments, the probability of assigning students to conditions changes while more data is being collected, so students can be assigned to interventions that are likely to perform better. Digital educational environments lower the barrier to conducting such adaptive experiments, but they are rarely applied in education. One reason might be that researchers have access to few real-world case studies that illustrate the advantages and disadvantages of these experiments in a specific context. We evaluate the effect of homework email reminders in students by conducting an adaptive experiment using the Thompson Sampling algorithm and compare it to a traditional uniform random experiment. We present this as a case study on how to conduct such experiments, and we raise a range of open questions about the conditions under which adaptive randomized experiments may be more or less useful.