Real-Time Multimodal Data Collection Using Smartwatches and Its Visualization in Education
This addresses the problem of limited adoption of Multimodal Learning Analytics in real-world educational settings for researchers and educators, though it is incremental as it builds on existing wearable technology.
The paper tackled the lack of scalable tools for multimodal data acquisition in education by developing Watch-DMLT and ViSeDOPS, enabling real-time monitoring and visualization of physiological and behavioral data from smartwatches, with a deployment involving 65 students and up to 16 devices demonstrating feasibility and utility.
Wearable sensors, such as smartwatches, have become increasingly prevalent across domains like healthcare, sports, and education, enabling continuous monitoring of physiological and behavioral data. In the context of education, these technologies offer new opportunities to study cognitive and affective processes such as engagement, attention, and performance. However, the lack of scalable, synchronized, and high-resolution tools for multimodal data acquisition continues to be a significant barrier to the widespread adoption of Multimodal Learning Analytics in real-world educational settings. This paper presents two complementary tools developed to address these challenges: Watch-DMLT, a data acquisition application for Fitbit Sense 2 smartwatches that enables real-time, multi-user monitoring of physiological and motion signals; and ViSeDOPS, a dashboard-based visualization system for analyzing synchronized multimodal data collected during oral presentations. We report on a classroom deployment involving 65 students and up to 16 smartwatches, where data streams including heart rate, motion, gaze, video, and contextual annotations were captured and analyzed. Results demonstrate the feasibility and utility of the proposed system for supporting fine-grained, scalable, and interpretable Multimodal Learning Analytics in real learning environments.