Towards A Human-in-the-Loop LLM Approach to Collaborative Discourse Analysis
This is an incremental step for educational researchers aiming to analyze collaborative learning, as it applies an existing method to a new domain.
The paper tackled the problem of characterizing synergistic learning in students' collaborative discourse by using a human-in-the-loop prompt engineering approach with GPT-4-Turbo, with preliminary findings suggesting it may perform comparably to humans.
LLMs have demonstrated proficiency in contextualizing their outputs using human input, often matching or beating human-level performance on a variety of tasks. However, LLMs have not yet been used to characterize synergistic learning in students' collaborative discourse. In this exploratory work, we take a first step towards adopting a human-in-the-loop prompt engineering approach with GPT-4-Turbo to summarize and categorize students' synergistic learning during collaborative discourse. Our preliminary findings suggest GPT-4-Turbo may be able to characterize students' synergistic learning in a manner comparable to humans and that our approach warrants further investigation.