Context Matters: Evaluating Context Strategies for Automated ADR Generation Using LLMs
This work addresses the burden of creating and maintaining ADRs for software developers, though it is incremental as it focuses on optimizing context strategies rather than introducing a new paradigm.
This paper tackles the problem of automating Architecture Decision Records (ADR) generation using Large Language Models (LLMs) to reduce authoring overhead, finding that context-aware prompting with a small recency window (3-5 prior records) substantially improves fidelity, while retrieval-based strategies offer only marginal gains in specific scenarios.
Architecture Decision Records (ADRs) play a critical role in preserving the rationale behind system design, yet their creation and maintenance are often neglected due to the associated authoring overhead. This paper investigates whether Large Language Models (LLMs) can mitigate this burden and, more importantly, how different strategies for presenting historical ADRs as context influence generation quality. We curate and validate a large corpus of sequential ADRs drawn from 750 open-source repositories and systematically evaluate five context selection strategies (no context, All-history, First-K, Last-K, and RAFG) across multiple model families. Our results show that context-aware prompting substantially improves ADR generation fidelity, with a small recency window (typically 3-5 prior records) providing the best balance between quality and efficiency. Retrieval-based context selection yields marginal gains primarily in non-sequential or cross-cutting decision scenarios, while offering no statistically significant advantage in typical linear ADR workflows. Overall, our findings demonstrate that context engineering, rather than model scale alone, is the dominant factor in effective ADR automation, and we outline practical defaults for tool builders along with targeted retrieval fallbacks for complex architectural settings.