How Do Students Interact with an LLM-powered Virtual Teaching Assistant in Different Educational Settings?
This research addresses the problem of adapting AI educational tools to different learning environments, though it is incremental in nature.
The study analyzed student interactions with an LLM-powered virtual teaching assistant named Jill Watson across various courses, finding that it encouraged higher-order cognitive questions but usage patterns varied significantly by context.
Jill Watson, a virtual teaching assistant powered by LLMs, answers student questions and engages them in extended conversations on courseware provided by the instructors. In this paper, we analyze student interactions with Jill across multiple courses and colleges, focusing on the types and complexity of student questions based on Bloom's Revised Taxonomy and tool usage patterns. We find that, by supporting a wide range of cognitive demands, Jill encourages students to engage in sophisticated, higher-order cognitive questions. However, the frequency of usage varies significantly across deployments, and the types of questions asked depend on course-specific contexts. These findings pave the way for future work on AI-driven educational tools tailored to individual learning styles and course structure, potentially enhancing both the teaching and learning experience in classrooms.