Anomaly Detection in Scratch Assignments
This work addresses the challenge for teachers of block-based programming languages like Scratch, who lack training in writing test suites, by providing an automated tool for bug detection, though it is incremental as it builds on prior research.
The paper tackles the problem of automatically detecting task-specific bugs in student Scratch assignments without requiring manual input from teachers, by enhancing a prior anomaly detection approach with a lower abstraction level to focus on relevant program parts.
For teachers, automated tool support for debugging and assessing their students' programming assignments is a great help in their everyday business. For block-based programming languages which are commonly used to introduce younger learners to programming, testing frameworks and other software analysis tools exist, but require manual work such as writing test suites or formal specifications. However, most of the teachers using languages like Scratch are not trained for or experienced in this kind of task. Linters do not require manual work but are limited to generic bugs and therefore miss potential task-specific bugs in student solutions. In prior work, we proposed the use of anomaly detection to find project-specific bugs in sets of student programming assignments automatically, without any additional manual labour required from the teachers' side. Evaluation on student solutions for typical programming assignments showed that anomaly detection is a reliable way to locate bugs in a data set of student programs. In this paper, we enhance our initial approach by lowering the abstraction level. The results suggest that the lower abstraction level can focus anomaly detection on the relevant parts of the programs.