Automated Feedback Generation for Introductory Programming Assignments
This addresses the challenge of providing scalable, personalized feedback for students in introductory programming courses, though it is incremental as it builds on existing automated feedback methods.
The paper tackles the problem of automatically generating feedback for introductory programming assignments by using a reference implementation and an error model to derive minimal corrections to student solutions, showing that simple error models can correct 65% of incorrect submissions on real student data.
We present a new method for automatically providing feedback for introductory programming problems. In order to use this method, we need a reference implementation of the assignment, and an error model consisting of potential corrections to errors that students might make. Using this information, the system automatically derives minimal corrections to student's incorrect solutions, providing them with a quantifiable measure of exactly how incorrect a given solution was, as well as feedback about what they did wrong. We introduce a simple language for describing error models in terms of correction rules, and formally define a rule-directed translation strategy that reduces the problem of finding minimal corrections in an incorrect program to the problem of synthesizing a correct program from a sketch. We have evaluated our system on thousands of real student attempts obtained from 6.00 and 6.00x. Our results show that relatively simple error models can correct on average 65% of all incorrect submissions.