Yanjie Zhao

Yanzhe Hu, Shenao Wang, Tianyuan Nie et al.

Large Language Models (LLMs) have revolutionized artificial intelligence (AI), driving breakthroughs in natural language understanding, text generation, and autonomous systems. However, the rapid growth of LLMs presents significant challenges in the security and reliability of the Large Language Model Supply Chain (LLMSC), a complex network of open-source components, libraries, and tools essential for LLM development and deployment. Despite its critical importance, the LLMSC remains underexplored, particularly regarding its structural characteristics, domain composition, and security vulnerabilities. To address this gap, we conduct the first empirical study of the LLMSC, analyzing a curated dataset of open-source packages from PyPI and NPM across 14 functional domains. We construct a directed dependency graph comprising 13,486 nodes, 28,704 edges, and 180 unique vulnerabilities to investigate the structural characteristics of the LLMSC and analyze how security risks propagate through its dependency network. Our findings reveal that the LLMSC exhibits a locally dense, globally sparse topology, with 72.38% of dependency trees containing fewer than 5 nodes, while a few large trees dominate the ecosystem, accounting for 77.66% of all nodes. The graph is characterized by high-degree hubs, with the top 5 most connected nodes averaging 1,207 dependents each. Security analysis shows that critical vulnerabilities propagate to an average of 142.1 nodes at the second layer of dependency trees and peak at 237.8 affected nodes at the third layer. Notably, cascading risks are concentrated in critical hub nodes such as \texttt{transformers}, which directly or indirectly affect over 1,300 downstream packages. These findings provide quantitative insights into the structural and security dynamics of the LLMSC and emphasize the need for targeted mitigation strategies to enhance ecosystem resilience.

Shenao Wang, Yanjie Zhao, Zhao Liu et al.

LLMs are rapidly transitioning from research prototypes to core components in production systems across industries such as finance and healthcare. These deployments rely on a growing ecosystem of open-source frameworks and components, collectively forming the LLM supply chain. However, the increasing complexity of this stack introduces critical security risks that remain underexplored. In this work, we present the first systematic and large-scale empirical study of vulnerabilities in the LLM supply chain, analyzing 529 real-world vulnerabilities spanning 77 widely adopted repositories across 12 lifecycle stages. Our findings reveal that the disclosed vulnerabilities are heavily concentrated in the application layer and model integration layer. Among these, 18.5% of the vulnerabilities are LLM-specific, arising from unique architectural and workflow characteristics, such as improper handling of critical resources like model files, prompt templates, and datasets, as well as generative output validation errors. To understand the real-world impact, we examine 63,243 publicly exposed LLM services and find that 45.6% are affected by at least one remotely exploitable vulnerability, over 70% of which are critical or high severity. By correlating these vulnerabilities with their potential exploit scenarios in the wild, we observed that these issues can lead to serious security consequences, including model tampering, sensitive dataset exposure, and unauthorized GPU resource abuse. Based on our findings, we distill 5 actionable insights that can guide engineering teams in auditing and securing LLM services. Our work offers a data-driven foundation for securing the LLM supply chain and highlights urgent directions for both industry and future research.

Hongzhou Rao, Yanjie Zhao, Xinyi Hou et al.

The rapid advancement of large language models (LLMs) has redefined artificial intelligence (AI), pushing the boundaries of AI research and enabling unbounded possibilities for both academia and the industry. However, LLM development faces increasingly complex challenges throughout its lifecycle, yet no existing research systematically explores these challenges and solutions from the perspective of software engineering (SE) approaches. To fill the gap, we systematically analyze research status throughout the LLM development lifecycle, divided into six phases: requirements engineering, dataset construction, model development and enhancement, testing and evaluation, deployment and operations, and maintenance and evolution. We then conclude by identifying the key challenges for each phase and presenting potential research directions to address these challenges. In general, we provide valuable insights from an SE perspective to facilitate future advances in LLM development.