CodeScholar: Growing Idiomatic Code Examples
This addresses the need for better API usage examples for programmers, offering a tool that improves documentation and code generation, though it is incremental as it builds on existing search and neural methods.
The paper tackles the problem of generating realistic and idiomatic code examples for API methods, presenting CodeScholar, a tool that developers prefer over GPT3.5 in 70% of cases and that produces more realistic, diverse, and concise examples across 85 queries from Python libraries.
Programmers often search for usage examples for API methods. A tool that could generate realistic, idiomatic, and contextual usage examples for one or more APIs would be immensely beneficial to developers. Such a tool would relieve the need for a deep understanding of the API landscape, augment existing documentation, and help discover interactions among APIs. We present CodeScholar, a tool that generates idiomatic code examples demonstrating the common usage of API methods. It includes a novel neural-guided search technique over graphs that grows the query APIs into idiomatic code examples. Our user study demonstrates that in 70% of cases, developers prefer CodeScholar generated examples over state-of-the-art large language models (LLM) like GPT3.5. We quantitatively evaluate 60 single and 25 multi-API queries from 6 popular Python libraries and show that across-the-board CodeScholar generates more realistic, diverse, and concise examples. In addition, we show that CodeScholar not only helps developers but also LLM-powered programming assistants generate correct code in a program synthesis setting.