C-Pack of IPAs: A C90 Program Benchmark of Introductory Programming Assignments
This provides a resource for researchers developing automated feedback tools for novice programmers in MOOCs, but it is incremental as it builds on existing benchmarks by adding specific program types.
The paper tackles the need for evaluating automated program repair techniques for introductory programming assignments by introducing C-Pack-IPAs, a benchmark containing 25 assignments with correct, incorrect, and syntactically flawed student programs and test suites.
Due to the vast number of students enrolled in Massive Open Online Courses (MOOCs), there has been an increasing number of automated program repair techniques focused on introductory programming assignments (IPAs). Such techniques take advantage of previous correct student implementations in order to provide automated, comprehensive, and personalized feedback to students. This paper presents C-Pack-IPAs, a publicly available benchmark of students' programs submitted for 25 different IPAs. C-Pack-IPAs contains semantically correct, semantically incorrect, and syntactically incorrect programs plus a test suite for each IPA. Hence, C-Pack-IPAs can be used to help evaluate the development of novel semantic, as well as syntactic, automated program repair frameworks, focused on providing feedback to novice programmers.