Cross-Subject Predictive Validity for Learning Outcomes of Delayed Start Behavior
For education researchers and practitioners, this work provides an interpretable, content-independent behavioral measure that aligns with teacher observations, though the effect sizes are small and the approach is incremental.
This paper examines the cross-subject predictive validity of delayed start behavior in educational software, finding that delayed starts during Math practice predict standardized test performance in both Math (β=.07 SD, p=.02) and English (β=.10 SD, p<.001). Using mixture modeling, they identify two sub-groups: early starters (<5 min delay, 20% of students) who show greater growth, and chronic delayers (>13 min delay, 20%) who show negative trends.
Behavioral detectors provide valuable insights into learner motivation and self-regulation. Among these, delayed start, a new session-level detector, has shown great promise as a valid behavioral measure that generalizes well across systems. In this paper, we examine cross-subject predictive validity of delayed start behavior. Using iReady data from 711 grade 7 students, we find delayed starts during Math practice are predictive of standardized test performance in both Math ($β$=.07 SD, p=.02) and English ($β$=.10 SD, p=<.001). Additionally, using mixture modeling and sensitivity analyses, we use a data-driven strategy to operationalize the identification of delayed starters in practice. We identify two underlying sub-groups of interest: "early starters" (<5 minute average delay, 20% of students) and "chronic delayers" (>13 minutes average delay, 20% of students). Relative to students in neither sub-group, early starters experienced greater growth (Math $β$=.11 SD, p=.07; ELA $β$=.15 SD, p=.02), while chronic delayers had the opposite trends (Math $β$=-.13 SD, p=0.05; ELA $β$=-.11 SD, p=0.11). Session-level measures provide a new opportunity for content-independent detectors, adding a behavioral component to the traditional usage and progress based on student engagement with content. This work aims to bridge education research with classroom practice by developing interpretable measures that align with behavioral cues teachers already use during classwork sessions to monitor and support students.