Generating Language Corrections for Teaching Physical Control Tasks
This addresses the challenge of offering personalized, language-based corrections for students in physical control education, though it is incremental as it builds on existing feedback systems by extending them to new domains.
The paper tackles the problem of providing detailed feedback for physical control tasks like drawing or steering, where current methods are limited to binary feedback or hand-coded templates, and shows that their model CORGI generates valid feedback, outperforms baselines on novel dynamics, and improves student learning in an interactive task.
AI assistance continues to help advance applications in education, from language learning to intelligent tutoring systems, yet current methods for providing students feedback are still quite limited. Most automatic feedback systems either provide binary correctness feedback, which may not help a student understand how to improve, or require hand-coding feedback templates, which may not generalize to new domains. This can be particularly challenging for physical control tasks, where the rich diversity in student behavior and specialized domains make it challenging to leverage general-purpose assistive tools for providing feedback. We design and build CORGI, a model trained to generate language corrections for physical control tasks, such as learning to ride a bike. CORGI takes in as input a pair of student and expert trajectories, and then generates natural language corrections to help the student improve. We collect and train CORGI over data from three diverse physical control tasks (drawing, steering, and joint movement). Through both automatic and human evaluations, we show that CORGI can (i) generate valid feedback for novel student trajectories, (ii) outperform baselines on domains with novel control dynamics, and (iii) improve student learning in an interactive drawing task.