TLDR: This research introduces HuggingGraph, a new method and graph model to understand the complex relationships between large language models (LLMs) and datasets on platforms like Hugging Face. By systematically collecting metadata and constructing a large, directed graph, the study reveals how LLMs are built upon each other and how datasets are reused. Key findings include the graph’s power-law distribution, a core of highly interconnected models and datasets, and the dynamic nature of the LLM ecosystem, with daily updates showing rapid growth in fine-tuned models. The work highlights the importance of tracing model origins and identifying critical components to improve fairness, security, and compliance in AI development.
Large language models, or LLMs, are at the forefront of artificial intelligence, capable of understanding and generating human-like text for a wide array of tasks, from translation to content creation. However, the sheer size and complexity involved in developing, training, and deploying these advanced models demand significant computational power and vast amounts of data. This often creates a hurdle for many researchers and developers.
In response to this challenge, platforms that host models and datasets have become incredibly popular. Hugging Face, for instance, has grown to host over 1.8 million models and more than 450,000 datasets by June 2025, and this growth shows no signs of slowing down.
The Challenge of Interconnectedness
A critical aspect of the LLM ecosystem is that new models are frequently built upon existing ‘base models’ or other pre-trained models, often utilizing external datasets. While this promotes efficiency and collaboration, it also means that vulnerabilities, biases, or even malicious components present in earlier models or datasets can be inherited. To address this, it becomes crucial to understand the origins and development processes of these components. This understanding is vital for detecting potential risks, enhancing model fairness, and ensuring compliance with various regulations.
Introducing HuggingGraph
Motivated by these challenges, a new project called HuggingGraph aims to systematically study the intricate relationships between models and datasets, which form the core of the LLM supply chain. The researchers designed a methodology to collect this supply chain information, primarily from Hugging Face, using its public APIs to gather metadata about hosted models and datasets. To overcome missing information, they employed techniques like cross-referencing links and extracting patterns from text descriptions.
With this collected data, HuggingGraph constructs a massive, directed, and heterogeneous graph. This graph, as of June 30, 2025, contains 397,376 nodes and 453,469 edges. In this graph, nodes represent different types of models (base, fine-tuned, adapter, quantization, and merge models) and datasets. Edges signify dependency relationships between them, including how datasets relate to each other (e.g., one being a subset of another) and how models depend on specific datasets for training.
Key Insights from the LLM Supply Chain
The analysis of the HuggingGraph revealed several fascinating insights into the LLM ecosystem:
-
Graph Properties: The LLM supply chain graph is large and sparse, exhibiting a ‘power-law’ distribution. This means that while most models and datasets have few connections, a small number of central ‘hub’ nodes—like certain base models and widely used datasets—have a vast number of connections. For example, the Mistral-7B-v0.1 model has the highest number of outgoing connections, indicating its widespread use.
-
Structural Patterns: The graph features a densely connected core, where major models like Gemma-2B and DistilBERT reside, facilitating efficient information sharing and reuse. Conversely, there’s a fragmented periphery of smaller, isolated components, often representing specialized or experimental projects. Community analysis further showed strong modularity, with models and datasets clustering around shared tasks, such as benchmarking or instruction-tuning.
-
Dataset Relationships: Datasets play a pivotal role. Some datasets, like ‘macrocosm-os/images’, act as comprehensive collections, including hundreds of smaller datasets. Others, like ‘HuggingFaceH4’, are foundational, being incorporated into nearly a thousand derived datasets. This highlights the extensive reuse and overlap within the dataset landscape.
-
Model-Dataset Interdependence: A strong mutual reliance exists between models and datasets. A single dataset, such as Mistral-7B-v0.1, can be used to train over a thousand different models, including fine-tuned, adapter, and quantized variants. Conversely, a single model, like DeBERTa-ST-AllLayers-v3.1, can be trained on a multitude of datasets (e.g., 116 different ones), showcasing its versatility across various tasks.
-
Temporal Evolution: The LLM ecosystem is highly dynamic. HuggingGraph employs a ‘Δ-based’ update mechanism to track daily changes efficiently. This tracking revealed that fine-tuned models dominate daily activity, with over 1,700 new entries per day on average, followed by adapters and datasets. This fine-grained temporal insight helps in understanding release bursts and evolving trends in the AI supply chain.
Also Read:
- Bridging the Divide: Understanding the State of Arabic LLM Training Data
- Large Language Models: A New Frontier in Cybersecurity
Practical Applications of HuggingGraph
The insights from HuggingGraph have significant practical implications. It can be used to trace the lineage and dependencies of LLMs, providing transparency about where a model originated, whether it inherited biases, or if it complies with licensing. This is crucial for developers, auditors, and policymakers to ensure responsible AI deployment. Furthermore, by identifying critical nodes—models or datasets that are highly reused—HuggingGraph helps in pinpointing potential single points of failure, allowing maintainers to safeguard vital assets and mitigate risks in the development pipeline.
This research offers a comprehensive look into the complex and rapidly evolving world of LLMs and their underlying data. For more details, you can refer to the full research paper: HuggingGraph: Understanding the Supply Chain of LLM Ecosystem.
