Credential Leakage in LLM Agent Skills: A Large-Scale Empirical Study
This addresses security risks for users of LLM agents by exposing widespread vulnerabilities in third-party skills, though it is incremental as it builds on existing security research with a new empirical focus.
The study tackled the problem of credential leakage in third-party skills for LLM agents, identifying 520 vulnerable skills with 1,708 issues and revealing that 76.3% of leaks require joint code and natural language analysis, with 89.6% of leaked credentials being exploitable without privileges.
Third-party skills extend LLM agents with powerful capabilities but often handle sensitive credentials in privileged environments, making leakage risks poorly understood. We present the first large-scale empirical study of this problem, analyzing 17,022 skills (sampled from 170,226 on SkillsMP) using static analysis, sandbox testing, and manual inspection. We identify 520 vulnerable skills with 1,708 issues and derive a taxonomy of 10 leakage patterns (4 accidental and 6 adversarial). We find that (1) leakage is fundamentally cross-modal: 76.3% require joint analysis of code and natural language, while 3.1% arise purely from prompt injection; (2) debug logging is the primary vector, with print and console.log causing 73.5% of leaks due to stdout exposure to LLMs; and (3) leaked credentials are both exploitable (89.6% without privileges) and persistent, as forks retain secrets even after upstream fixes. After disclosure, all malicious skills were removed and 91.6% of hardcoded credentials were fixed. We release our dataset, taxonomy, and detection pipeline to support future research.