Linting Style and Substance in READMEs
This work addresses the need for better documentation linting in software development, though it is incremental as it builds on existing linting concepts with novel integrations.
The paper tackled the problem of improving READMEs for software projects by developing LintMe, a tool that uses a domain-specific language to create context-specific checks combining programmatic operations and LLM-based content evaluation. The result showed that the design is approachable and flexible, as demonstrated in a user study with 11 participants.
READMEs shape first impressions of software projects, yet what constitutes a good README varies across audiences and contexts. Research software needs reproducibility details, while open-source libraries might prioritize quick-start guides. Through a design probe, LintMe, we explore how linting can be used to improve READMEs given these diverse contexts, aiding style and content issues while preserving authorial agency. Users create context-specific checks using a lightweight DSL that uses a novel combination of programmatic operations (e.g., for broken links) with LLM-based content evaluation (e.g., for detecting jargon), yielding checks that would be challenging for prior linters. Through a user study (N=11), comparison with naive LLM usage, and an extensibility case study, we find that our design is approachable, flexible, and well matched with the needs of this domain. This work opens the door for linting more complex documentation and other culturally mediated text-based documents.