Sketch Bug: Using Sketch-Based Input for Interactive Code Debugging
For programmers, this work provides an alternative debugging interaction method, though it is incremental as it primarily extends existing gesture recognition to debugging without major performance gains.
The paper explores using sketch-based pen input for interactive code debugging, finding it can support execution control tasks like breakpoint setting and step-wise execution, but introduces challenges in precision and gesture recall. A study with 24 programmers showed sketch input is most beneficial for spatially grounded or continuous movement interactions.
We investigate sketch-like pen input as an alternative way to support execution control in interactive debugging. In our interface, programmers draw lightweight marks to set breakpoints, use symbolic strokes to control execution, and extend strokes into spirals to repeat traversal actions. The prototype combines gesture recognition with Python execution tracing in a conventional editor interface. In a controlled study with 24 programmers, we compared the sketch interface with conventional mouse-and-keyboard input on debugging tasks that required breakpoint placement, step-wise execution, and runtime state inspection. The results show that sketch-like input can support these execution-control tasks, while also introducing challenges in precision, recognition, and gesture recall. Our findings suggest that pen input is most promising where debugger interactions benefit from spatial grounding or continuous movement, rather than as a wholesale replacement for conventional debugging controls.