SnapCheck: Automated Testing for Snap Programs
This addresses the challenge for instructors and researchers in providing automated feedback to students in media-focused programming environments, though it is incremental as it builds on existing testing concepts for a specific domain.
The paper tackles the problem of assessing visual, interactive programming projects like those in Snap, which is time-consuming and manual, by introducing SnapCheck, a dynamic testing framework that achieves at least 98% accuracy in automated testing on a Pong game assignment.
Programming environments such as Snap, Scratch, and Processing engage learners by allowing them to create programming artifacts such as apps and games, with visual and interactive output. Learning programming with such a media-focused context has been shown to increase retention and success rate. However, assessing these visual, interactive projects requires time and laborious manual effort, and it is therefore difficult to offer automated or real-time feedback to students as they work. In this paper, we introduce SnapCheck, a dynamic testing framework for Snap that enables instructors to author test cases with Condition-Action templates. The goal of SnapCheck is to allow instructors or researchers to author property-based test cases that can automatically assess students' interactive programs with high accuracy. Our evaluation of SnapCheck on 162 code snapshots from a Pong game assignment in an introductory programming course shows that our automated testing framework achieves at least 98% accuracy over all rubric items, showing potentials to use SnapCheck for auto-grading and providing formative feedback to students.