Active Learning for Efficient Testing of Student Programs
This addresses the practical need for faster automated testing of student code submissions in educational settings, though it appears incremental as it builds upon existing symbolic execution techniques.
The researchers tackled the problem of efficiently identifying semantic bugs in student programs by developing ATAS, a method combining symbolic execution with active learning that achieved over 2.5x runtime improvement compared to a baseline while maintaining evaluation accuracy.
In this work, we propose an automated method to identify semantic bugs in student programs, called ATAS, which builds upon the recent advances in both symbolic execution and active learning. Symbolic execution is a program analysis technique which can generate test cases through symbolic constraint solving. Our method makes use of a reference implementation of the task as its sole input. We compare our method with a symbolic execution-based baseline on 6 programming tasks retrieved from CodeForces comprising a total of 23K student submissions. We show an average improvement of over 2.5x over the baseline in terms of runtime (thus making it more suitable for online evaluation), without a significant degradation in evaluation accuracy.