Tutorly: Turning Programming Videos Into Apprenticeship Learning Environments with LLMs
This addresses the problem of passive learning from programming videos for learners by providing tailored guidance and feedback, though it is incremental as it builds on existing cognitive apprenticeship and LLM frameworks.
The authors tackled the challenge of making online programming videos more interactive and personalized by developing Tutorly, a JupyterLab plugin that uses LLMs to turn videos into one-on-one tutoring experiences, resulting in a significant performance improvement from 61.9% to 76.6% in a study with 16 participants.
Online programming videos, including tutorials and streamcasts, are widely popular and contain a wealth of expert knowledge. However, effectively utilizing these resources to achieve targeted learning goals can be challenging. Unlike direct tutoring, video content lacks tailored guidance based on individual learning paces, personalized feedback, and interactive engagement necessary for support and monitoring. Our work transforms programming videos into one-on-one tutoring experiences using the cognitive apprenticeship framework. Tutorly, developed as a JupyterLab Plugin, allows learners to (1) set personalized learning goals, (2) engage in learning-by-doing through a conversational LLM-based mentor agent, (3) receive guidance and feedback based on a student model that steers the mentor moves. In a within-subject study with 16 participants learning exploratory data analysis from a streamcast, Tutorly significantly improved their performance from 61.9% to 76.6% based on a post-test questionnaire. Tutorly demonstrates the potential for enhancing programming video learning experiences with LLM and learner modeling.