18.5AIMay 7
Towards Security-Auditable LLM Agents: A Unified Graph RepresentationChaofan Li, Lyuye Zhang, Jintao Zhai et al.
LLM-based agentic systems are rapidly evolving to perform complex autonomous tasks through dynamic tool invocation, stateful memory management, and multi-agent collaboration. However, this semantics-driven execution paradigm creates a severe semantic gap between low-level physical events and high-level execution intent, making post-hoc security auditing fundamentally difficult. Existing representation mechanisms, including static SBOMs and runtime logs, provide only fragmented evidence and fail to capture cognitive-state evolution, capability bindings, persistent memory contamination, and cascading risk propagation across interacting agents. To bridge this gap, we propose Agent-BOM, a unified structural representation for agent security auditing. Agent-BOM models an agentic system as a hierarchical attributed directed graph that separates static capability bases, such as models, tools, and long-term memory, from dynamic runtime semantic states, such as goals, reasoning trajectories, and actions. These layers are connected through semantic edges and security attributes, transforming fragmented execution traces into queryable audit paths. Building on Agent-BOM, we develop a graph-query-based paradigm for path-level risk assessment and instantiate it with the OWASP Agentic Top 10. We further implement an auditing plugin in the OpenClaw environment to construct Agent-BOM from live executions. Evaluation on representative real-world agentic attack scenarios shows that Agent-BOM can reconstruct stealthy attack chains, including cross-session memory poisoning and tool misuse, capability supply-chain hijacking and unexpected code execution, multi-agent ecosystem hijacking, and privilege and trust abuse. These results demonstrate that Agent-BOM provides a unified and auditable foundation for root-cause analysis and security adjudication in complex agentic ecosystems.
9.3SEMar 23
SkillClone: Multi-Modal Clone Detection and Clone Propagation Analysis in the Agent Skill EcosystemJiaying Zhu, Lyuye Zhang, Wenbo Guo et al.
Agent skills are modular instruction packages that combine YAML metadata, natural language instructions, and embedded code, and they have reached 196K publicly available instances, yet no mechanism exists to detect clone relationships among them. This gap creates systemic risks: a vulnerability in a widely copied skill silently persists across derivatives with no alert to maintainers. Existing clone detectors, designed for single-modality source code, cannot handle the multi-modal structure of skills, where clone evidence is distributed across three interleaved content channels. We present SkillClone, the first multi-modal clone detection approach for agent skills. SkillClone fuses flat TF-IDF similarity with per-channel decomposition (YAML, NL, code) through logistic regression, combining strong detection with interpretable type classification. We construct SkillClone-Bench, a balanced benchmark of 300 ground-truth pairs with stratified difficulty. On SkillClone-Bench, SkillClone achieves F1 of 0.939 with precision 0.952, outperforming flat TF-IDF (F1 = 0.881) and achieving 4.2x higher Type-4 (semantic) recall than MinHash. Applying SkillClone to 20K skills reveals 258K clone pairs involving 75% of all skills, with 40% crossing author boundaries. A deduplication analysis shows the ecosystem is inflated 3.5x: only 5,642 unique skill concepts underlie the 20K listed skills, and 41% of skills in clone families are superseded by a strictly better variant.
30.0CRJan 29, 2024
LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs' Vulnerability ReasoningYuqiang Sun, Daoyuan Wu, Yue Xue et al.
Large language models (LLMs) have demonstrated significant potential in various tasks, including those requiring human-level intelligence, such as vulnerability detection. However, recent efforts to use LLMs for vulnerability detection remain preliminary, as they lack a deep understanding of whether a subject LLM's vulnerability reasoning capability stems from the model itself or from external aids such as knowledge retrieval and tooling support. In this paper, we aim to decouple LLMs' vulnerability reasoning from other capabilities, such as vulnerability knowledge adoption, context information retrieval, and advanced prompt schemes. We introduce LLM4Vuln, a unified evaluation framework that separates and assesses LLMs' vulnerability reasoning capabilities and examines improvements when combined with other enhancements. To support this evaluation, we construct UniVul, the first benchmark that provides retrievable knowledge and context-supplementable code across three representative programming languages: Solidity, Java, and C/C++. Using LLM4Vuln and UniVul, we test six representative LLMs (GPT-4.1, Phi-3, Llama-3, o4-mini, DeepSeek-R1, and QwQ-32B) for 147 ground-truth vulnerabilities and 147 non-vulnerable cases in 3,528 controlled scenarios. Our findings reveal the varying impacts of knowledge enhancement, context supplementation, and prompt schemes. We also identify 14 zero-day vulnerabilities in four pilot bug bounty programs, resulting in $3,576 in bounties.