Understanding Prompt Management in GitHub Repositories: A Call for Best Practices
This addresses prompt management problems for developers in the promptware ecosystem, but is incremental as it identifies issues without proposing novel solutions.
The study analyzed 24,800 open-source prompts from 92 GitHub repositories to investigate prompt management practices, revealing critical challenges including inconsistencies in formatting, substantial duplication, and frequent readability and spelling issues.
The rapid adoption of foundation models (e.g., large language models) has given rise to promptware, i.e., software built using natural language prompts. Effective management of prompts, such as organization and quality assurance, is essential yet challenging. In this study, we perform an empirical analysis of 24,800 open-source prompts from 92 GitHub repositories to investigate prompt management practices and quality attributes. Our findings reveal critical challenges such as considerable inconsistencies in prompt formatting, substantial internal and external prompt duplication, and frequent readability and spelling issues. Based on these findings, we provide actionable recommendations for developers to enhance the usability and maintainability of open-source prompts within the rapidly evolving promptware ecosystem.