RAPGen: An Approach for Fixing Code Inefficiencies in Zero-Shot
This addresses the challenge of performance bug fixing for developers, offering a zero-shot method that is incremental by building on existing prompt and retrieval techniques.
The paper tackles the problem of fixing performance bugs in code by introducing RAPGen, a retrieval-augmented prompt generation approach that uses a knowledge-base of past fixes to generate prompts for a large language model, achieving performance improvements equivalent or better than developers in about 60% of cases, with 42% verbatim matches in an expert-verified dataset.
Performance bugs are non-functional bugs that can even manifest in well-tested commercial products. Fixing these performance bugs is an important yet challenging problem. In this work, we address this challenge and present a new approach called Retrieval-Augmented Prompt Generation (RAPGen). Given a code snippet with a performance issue, RAPGen first retrieves a prompt instruction from a pre-constructed knowledge-base of previous performance bug fixes and then generates a prompt using the retrieved instruction. It then uses this prompt on a Large Language Model (such as Codex) in zero-shot to generate a fix. We compare our approach with the various prompt variations and state of the art methods in the task of performance bug fixing. Our evaluation shows that RAPGen can generate performance improvement suggestions equivalent or better than a developer in ~60% of the cases, getting ~42% of them verbatim, in an expert-verified dataset of past performance changes made by C# developers.