LLMs as Educational Analysts: Transforming Multimodal Data Traces into Actionable Reading Assessment Reports
This addresses the need for more detailed reading assessment insights for educators, though it is incremental as it builds on existing data analytics with LLMs.
This study tackled the problem of limited insights from reading assessments by using multimodal data like eye-tracking and learning outcomes to generate actionable reports for educators, finding that LLMs can effectively synthesize this data into teacher-friendly insights that are well-received.
Reading assessments are essential for enhancing students' comprehension, yet many EdTech applications focus mainly on outcome-based metrics, providing limited insights into student behavior and cognition. This study investigates the use of multimodal data sources -- including eye-tracking data, learning outcomes, assessment content, and teaching standards -- to derive meaningful reading insights. We employ unsupervised learning techniques to identify distinct reading behavior patterns, and then a large language model (LLM) synthesizes the derived information into actionable reports for educators, streamlining the interpretation process. LLM experts and human educators evaluate these reports for clarity, accuracy, relevance, and pedagogical usefulness. Our findings indicate that LLMs can effectively function as educational analysts, turning diverse data into teacher-friendly insights that are well-received by educators. While promising for automating insight generation, human oversight remains crucial to ensure reliability and fairness. This research advances human-centered AI in education, connecting data-driven analytics with practical classroom applications.