Machine Learning Analysis of Heterogeneity in the Effect of Student Mindset Interventions
This work addresses the need for personalized educational interventions by identifying moderators of treatment effects, though it is incremental in applying existing ML methods to a specific dataset.
The study tackled the problem of understanding how a mindset intervention affects student performance differently across individuals, finding an average positive effect of 0.26 with heterogeneity influenced by school-level factors like achievement and poverty.
We study heterogeneity in the effect of a mindset intervention on student-level performance through an observational dataset from the National Study of Learning Mindsets (NSLM). Our analysis uses machine learning (ML) to address the following associated problems: assessing treatment group overlap and covariate balance, imputing conditional average treatment effects, and interpreting imputed effects. By comparing several different model families we illustrate the flexibility of both off-the-shelf and purpose-built estimators. We find that the mindset intervention has a positive average effect of 0.26, 95%-CI [0.22, 0.30], and that heterogeneity in the range of [0.1, 0.4] is moderated by school-level achievement level, poverty concentration, urbanicity, and student prior expectations.