Synthesize, Execute and Debug: Learning to Repair for Neural Program Synthesis
This work addresses program synthesis for AI systems, offering an incremental improvement by incorporating debugging to enhance neural synthesizers.
The authors tackled the challenge of synthesizing programs consistent with complex specifications from input-output examples by proposing SED, a neural framework that integrates synthesis, execution, and debugging stages, which reduced error rates on the Karel benchmark.
The use of deep learning techniques has achieved significant progress for program synthesis from input-output examples. However, when the program semantics become more complex, it still remains a challenge to synthesize programs that are consistent with the specification. In this work, we propose SED, a neural program generation framework that incorporates synthesis, execution, and debugging stages. Instead of purely relying on the neural program synthesizer to generate the final program, SED first produces initial programs using the neural program synthesizer component, then utilizes a neural program debugger to iteratively repair the generated programs. The integration of the debugger component enables SED to modify the programs based on the execution results and specification, which resembles the coding process of human programmers. On Karel, a challenging input-output program synthesis benchmark, SED reduces the error rate of the neural program synthesizer itself by a considerable margin, and outperforms the standard beam search for decoding.