Fix Bugs with Transformer through a Neural-Symbolic Edit Grammar
This addresses the problem of automated code repair for software developers, representing an incremental improvement with specific gains.
The paper tackles the problem of automatically fixing bugs in source code by introducing NSEdit, a Transformer-based method that predicts editing sequences, achieving a new state-of-the-art accuracy of 24.04% on the Tufano small dataset of the CodeXGLUE benchmark.
We introduce NSEdit (neural-symbolic edit), a novel Transformer-based code repair method. Given only the source code that contains bugs, NSEdit predicts an editing sequence that can fix the bugs. The edit grammar is formulated as a regular language, and the Transformer uses it as a neural-symbolic scripting interface to generate editing programs. We modify the Transformer and add a pointer network to select the edit locations. An ensemble of rerankers are trained to re-rank the editing sequences generated by beam search. We fine-tune the rerankers on the validation set to reduce over-fitting. NSEdit is evaluated on various code repair datasets and achieved a new state-of-the-art accuracy ($24.04\%$) on the Tufano small dataset of the CodeXGLUE benchmark. NSEdit performs robustly when programs vary from packages to packages and when buggy programs are concrete. We conduct detailed analysis on our methods and demonstrate the effectiveness of each component.