Combining model tracing and constraint-based modeling for multistep strategy diagnoses
This work addresses the challenge of accurate multistep strategy diagnosis in educational technology, though it appears incremental as it combines existing paradigms.
The authors tackled the problem of diagnosing student input in stepwise tasks by merging model tracing and constraint-based modeling, resulting in a system that aligned with teacher coding in all 140 evaluated student steps.
Model tracing and constraint-based modeling are two approaches to diagnose student input in stepwise tasks. Model tracing supports identifying consecutive problem-solving steps taken by a student, whereas constraint-based modeling supports student input diagnosis even when several steps are combined into one step. We propose an approach that merges both paradigms. By defining constraints as properties that a student input has in common with a step of a strategy, it is possible to provide a diagnosis when a student deviates from a strategy even when the student combines several steps. In this study we explore the design of a system for multistep strategy diagnoses, and evaluate these diagnoses. As a proof of concept, we generate diagnoses for an existing dataset containing steps students take when solving quadratic equations (n=2136). To compare with human diagnoses, two teachers coded a random sample of deviations (n=70) and applications of the strategy (n=70). Results show that that the system diagnosis aligned with the teacher coding in all of the 140 student steps.