TraceDiff: Debugging Unexpected Code Behavior Using Trace Divergences
This work addresses the problem of providing scalable, personalized feedback in massive programming classrooms, though it is incremental as it builds on existing program synthesis and visualization techniques.
The paper tackles the challenge of generating pedagogically effective automated feedback for programming assignments by analyzing teacher practices to establish design guidelines and developing a system that compares dynamic code execution traces to highlight errors. Results show that their system helps students detect and fix bugs more effectively than an existing visual debugging tool.
Recent advances in program synthesis offer means to automatically debug student submissions and generate personalized feedback in massive programming classrooms. When automatically generating feedback for programming assignments, a key challenge is designing pedagogically useful hints that are as effective as the manual feedback given by teachers. Through an analysis of teachers' hint-giving practices in 132 online Q&A posts, we establish three design guidelines that an effective feedback design should follow. Based on these guidelines, we develop a feedback system that leverages both program synthesis and visualization techniques. Our system compares the dynamic code execution of both incorrect and fixed code and highlights how the error leads to a difference in behavior and where the incorrect code trace diverges from the expected solution. Results from our study suggest that our system enables students to detect and fix bugs that are not caught by students using another existing visual debugging tool.