Through the WordStream Glass: Revisiting Quantitative Encoding for Qualitative Learning Analytics
This work provides design implications for visualization tools used by education researchers, addressing the gap between quantitative learning analytics and qualitative inquiry.
The study examined how qualitative education researchers use two WordStream-based visualization platforms for analyzing student journal responses, revealing an epistemological tension where frequency-based visualizations are seen as both a productive entry point and a potential obscurer of rare but critical responses. The findings inform design implications for tools that better integrate quantitative and qualitative analysis.
Data-driven learning analytics can surface trends across a student cohort over time, helping instructors improve the learning environment. WordStream, a visualization idiom for topic evolution, has been instantiated in two platforms toward this goal: the Journal Data Dashboard, for analyzing formative assessments, and WordStream Maker, for authoring custom visualizations. Where the prior work built these platforms for education (Vis4Ed), here we examine the reverse direction (Ed4Vis): what can qualitative education research tell us about building better visualization tools? We conducted a mixed-methods expert study (n=10) in which STEM education researchers with expertise in qualitative methods and classroom assessment used both platforms to analyze student journal responses from a data visualization course. Across two cycles of thematic analysis with confirmatory checking, we report themes spanning tool experience, disciplinary context of use, and, most importantly, a core epistemological dissensus. Some instructor-researchers regarded frequency-based visualization as a productive entry point to qualitative analysis; others cautioned it can obscure rare but critical responses. We synthesize these findings into design implications for future tools that better integrate quantitative technique with qualitative inquiry. All Supplementary Materials are available at https://osf.io/z2f8d.