Few-shot training LLMs for project-specific code-summarization
This addresses the challenge of limited project-specific data in software engineering for code summarization, though it appears incremental as it applies an existing few-shot method to a new domain.
The paper tackles the problem of code summarization by leveraging few-shot training with the GPT Codex model on project-specific data, achieving results that significantly surpass state-of-the-art models.
Very large language models (LLMs), such as GPT-3 and Codex have achieved state-of-the-art performance on several natural-language tasks, and show great promise also for code. A particularly exciting aspect of LLMs is their knack for few-shot and zero-shot learning: they can learn to perform a task with very few examples. Few-shotting has particular synergies in software engineering, where there are a lot of phenomena (identifier names, APIs, terminology, coding patterns) that are known to be highly project-specific. However, project-specific data can be quite limited, especially early in the history of a project; thus the few-shot learning capacity of LLMs might be very relevant. In this paper, we investigate the use few-shot training with the very large GPT (Generative Pre-trained Transformer) Codex model, and find evidence suggesting that one can significantly surpass state-of-the-art models for code-summarization, leveraging project-specific training.