The Hitchhiker's Guide to Program Analysis, Part II: Deep Thoughts by LLMs
This addresses the problem of excessive false positives in static analysis tools for software security, particularly in large codebases like the Linux kernel, representing an incremental improvement by enhancing existing methods with LLM-based refinement.
The paper tackles the precision-scalability tradeoff in static analysis for software vulnerability detection by introducing BugLens, a post-refinement framework that uses LLMs to assess security impact and validate constraints, improving precision approximately 7-fold from 0.10 to 0.72 on Linux kernel taint-style bugs and uncovering four previously unreported vulnerabilities.
Static analysis plays a crucial role in software vulnerability detection, yet faces a persistent precision-scalability tradeoff. In large codebases like the Linux kernel, traditional static analysis tools often generate excessive false positives due to simplified vulnerability modeling and overapproximation of path and data constraints. While large language models (LLMs) demonstrate promising code understanding capabilities, their direct application to program analysis remains unreliable due to inherent reasoning limitations. We introduce BugLens, a post-refinement framework that significantly enhances static analysis precision for bug detection. BugLens guides LLMs through structured reasoning steps to assess security impact and validate constraints from the source code. When evaluated on Linux kernel taint-style bugs detected by static analysis tools, BugLens improves precision approximately 7-fold (from 0.10 to 0.72), substantially reducing false positives while uncovering four previously unreported vulnerabilities. Our results demonstrate that a well-structured, fully automated LLM-based workflow can effectively complement and enhance traditional static analysis techniques.