Modeling Collaborative Problem Solving Dynamics from Group Discourse: A Text-Mining Approach with Synergy Degree Model
For learning analytics researchers, this work provides a scalable, AI-in-the-loop method to measure CPS synergy automatically, addressing the bottleneck of manual coding.
This study introduces a computational framework integrating automated discourse analysis with the Synergy Degree Model (SDM) to quantify collaborative problem solving (CPS) synergy from group communication. Results show that synergy degree effectively distinguishes collaborative quality (from excellent to failing groups) and reveals significant task-type differences, with survey study groups exhibiting higher creation-order than mode study groups.
Measuring collaborative problem solving (CPS) synergy remains challenging in learning analytics, as classical manual coding cannot capture emergent system-level dynamics. This study introduces a computational framework that integrates automated discourse analysis with the Synergy Degree Model (SDM) to quantify CPS synergy from group communication. Data were collected from 52 learners in 12 groups during a 5-week connectivist MOOC (cMOOC) activity. Nine classification models were applied to automatically identify ten CPS behaviors across four interaction levels: operation, wayfinding, sense-making, and creation. While BERT achieved the highest accuracy, GPT models demonstrated superior precision suitable for human-AI collaborative coding. Within the SDM framework, each interaction level was treated as a subsystem to compute group-level order parameters and derive synergy degrees. Permutation tests showed automated measures preserve construct validity, despite systematic biases at the subsystem level. Statistical analyses revealed significant task-type differences: survey study groups exhibited higher creation-order than mode study groups, suggesting "controlled disorder" may benefit complex problem solving. Importantly, synergy degree distinguished collaborative quality, ranging from excellent to failing groups. Findings establish synergy degree as a sensitive indicator of collaboration and demonstrate the feasibility of scaling fine-grained CPS analytics through AI-in-the-loop approaches.