Taint-Style Vulnerability Detection and Confirmation for Node.js Packages Using LLM Agent Reasoning
This addresses a critical security issue for the Node.js ecosystem, which includes millions of packages in software supply chains, by offering a novel approach that outperforms traditional methods without requiring vulnerability annotations or reports.
The paper tackles the problem of detecting and confirming taint-style vulnerabilities in Node.js packages by developing LLMVD.js, a multi-stage LLM agent pipeline that scans code, proposes vulnerabilities, generates exploits, and validates them; it confirms 84% of vulnerabilities on public benchmarks compared to less than 22% for prior tools and generates validated exploits for 36 out of 260 recently released packages.
The rapidly evolving Node$.$js ecosystem currently includes millions of packages and is a critical part of modern software supply chains, making vulnerability detection of Node$.$js packages increasingly important. However, traditional program analysis struggles in this setting because of dynamic JavaScript features and the large number of package dependencies. Recent advances in large language models (LLMs) and the emerging paradigm of LLM-based agents offer an alternative to handcrafted program models. This raises the question of whether an LLM-centric, tool-augmented approach can effectively detect and confirm taint-style vulnerabilities (e.g., arbitrary command injection) in Node$.$js packages. We implement LLMVD$.$js, a multi-stage agent pipeline to scan code, propose vulnerabilities, generate proof-of-concept exploits, and validate them through lightweight execution oracles; and systematically evaluate its effectiveness in taint-style vulnerability detection and confirmation in Node$.$js packages without dedicated static/dynamic analysis engines for path derivation. For packages from public benchmarks, LLMVD$.$js confirms 84% of the vulnerabilities, compared to less than 22% for prior program analysis tools. It also outperforms a prior LLM-program-analysis hybrid approach while requiring neither vulnerability annotations nor prior vulnerability reports. When evaluated on a set of 260 recently released packages (without vulnerability groundtruth information), traditional tools produce validated exploits for few ($\leq 2$) packages, while LLMVD$.$js generates validated exploits for 36 packages.