Paper Information
- Author
- Yujie Ma,Lili Quan,Xiaofei Xie,Qiang Hu,Jiongchi Yu,Yao Zhang,Sen Chen
- Published
- 7-24-2025
- Affiliation
- Tianjin University
- Country
- China
- Conference
- Computing Research Repository (CoRR)
Abstract
The rise of Large Language Models (LLMs) has led to the widespread deployment
of LLM-based systems across diverse domains. As these systems proliferate,
understanding the risks associated with their complex supply chains is
increasingly important. LLM-based systems are not standalone as they rely on
interconnected supply chains involving pretrained models, third-party
libraries, datasets, and infrastructure. Yet, most risk assessments narrowly
focus on model or data level, overlooking broader supply chain vulnerabilities.
While recent studies have begun to address LLM supply chain risks, there
remains a lack of benchmarks for systematic research.
To address this gap, we introduce the first comprehensive dataset for
analyzing and benchmarking LLM supply chain security. We collect 3,859
real-world LLM applications and perform interdependency analysis, identifying
109,211 models, 2,474 datasets, and 9,862 libraries. We extract model
fine-tuning paths, dataset reuse, and library reliance, mapping the ecosystem's
structure. To evaluate security, we gather 1,555 risk-related issues-50 for
applications, 325 for models, 18 for datasets, and 1,229 for libraries from
public vulnerability databases.
Using this dataset, we empirically analyze component dependencies and risks.
Our findings reveal deeply nested dependencies in LLM applications and
significant vulnerabilities across the supply chain, underscoring the need for
comprehensive security analysis. We conclude with practical recommendations to
guide researchers and developers toward safer, more trustworthy LLM-enabled
systems.