SYNFIX: Automatically Fixing Syntax Errors using Compiler Diagnostics
This addresses the issue of cryptic compiler error messages for beginning programmers, potentially saving instructors' time, though it is incremental as it builds on existing machine-learning approaches.
The paper tackles the problem of syntax errors in programming for beginners by introducing SYNFIX, a tool that uses machine learning to automatically fix these errors, achieving substantial improvements over state-of-the-art methods.
Beginning programmers struggle with the complex grammar of modern programming languages like Java, and make lot of syntax errors. The diagnostic syntax error messages from compilers and IDEs are sometimes useful, but often the messages are cryptic and puzzling. Students could be helped, and instructors' time saved, by automated repair suggestions when dealing with syntax errors. Large samples of student errors and fixes are now available, offering the possibility of data-driven machine-learning approaches to help students fix syntax errors. Current machine-learning approaches do a reasonable job fixing syntax errors in shorter programs, but don't work as well even for moderately longer programs. We introduce SYNFIX, a machine-learning based tool that substantially improves on the state-of-the-art, by learning to use compiler diagnostics, employing a very large neural model that leverages unsupervised pre-training, and relying on multi-label classification rather than autoregressive synthesis to generate the (repaired) output. We describe SYNFIX's architecture in detail, and provide a detailed evaluation. We have built SYNFIX into a free, open-source version of Visual Studio Code; we make all our source code and models freely available.