VisDocSketcher: Towards Scalable Visual Documentation with Agentic Systems
This addresses the problem of time-consuming and subjective visual documentation creation for developers working with large software systems, representing a novel foundational contribution rather than an incremental improvement.
This paper tackles the problem of automatically generating visual documentation from code, which is difficult to produce and evaluate manually, by introducing VisDocSketcher, the first agent-based approach that combines static analysis with LLM agents. The results show it generates valid visual documentation for 74.4% of samples, improving 26.7-39.8% over a baseline, and includes an evaluation framework achieving an AUC exceeding 0.87.
Visual documentation is an effective tool for reducing the cognitive barrier developers face when understanding unfamiliar code, enabling more intuitive comprehension. Compared to textual documentation, it provides a higher-level understanding of the system structure and data flow. Developers usually prefer visual representations over lengthy textual descriptions for large software systems. Visual documentation is both difficult to produce and challenging to evaluate. Manually creating it is time-consuming, and currently, no existing approach can automatically generate high-level visual documentation directly from code. Its evaluation is often subjective, making it difficult to standardize and automate. To address these challenges, this paper presents the first exploration of using agentic LLM systems to automatically generate visual documentation. We introduce VisDocSketcher, the first agent-based approach that combines static analysis with LLM agents to identify key elements in the code and produce corresponding visual representations. We propose a novel evaluation framework, AutoSketchEval, for assessing the quality of generated visual documentation using code-level metrics. The experimental results show that our approach can valid visual documentation for 74.4% of the samples. It shows an improvement of 26.7-39.8% over a simple template-based baseline. Our evaluation framework can reliably distinguish high-quality (code-aligned) visual documentation from low-quality (non-aligned) ones, achieving an AUC exceeding 0.87. Our work lays the foundation for future research on automated visual documentation by introducing practical tools that not only generate valid visual representations but also reliably assess their quality.