TLDR: The Fabric project introduces a public repository of 20 real-world AI use cases, detailing their practical governance mechanisms. Through interviews and co-designed diagrams with practitioners, the research identifies four levels of human oversight (Autonomous, Conditionally Autonomous, Human-Approved, Human-Led with AI-Assistance) and five levels of institutional oversight (Ad-Hoc Practice, Organization Best Practice, Organization Policy, Industry Standard, Regulation). The findings highlight common patterns, the prevalence of human-led AI assistance, and the importance of regulatory compliance, while also pointing to gaps in clear AI governance guidelines and challenges with third-party systems. Fabric aims to provide empirical insights into AI governance to inform future practices.
As artificial intelligence becomes more deeply integrated into our daily lives, from managing finances to assisting with creative writing, a crucial question arises: how are these AI systems actually governed in the real world? While much academic discussion has focused on the risks and harms of AI, there’s been a noticeable gap in understanding the practical mechanisms that make AI deployment safe and possible.
A new initiative called Fabric aims to bridge this gap. Fabric is a publicly available repository that documents real-world AI use cases and, crucially, outlines the governance mechanisms applied to them. This project moves beyond theoretical discussions to show how oversight and safeguards are implemented in practice.
How Fabric Was Built
To create this unique repository, the researchers behind Fabric conducted semi-structured interviews with AI practitioners – individuals directly involved in developing, deploying, or maintaining AI systems within various organizations. During these interviews, they didn’t just talk; they co-designed visual diagrams of the AI workflow with the practitioners. This collaborative approach helped to clearly map out how AI systems operate and where governance is applied. The initial release of Fabric includes 20 such AI use cases, spanning different sectors like public services, private industry, and civil society.
Understanding AI Oversight: Two Key Dimensions
The analysis of these real-world examples revealed two primary types of oversight: human oversight and institutional oversight. These two dimensions combine to form what the researchers call “Patterns of Oversight.”
Human Oversight Levels: Who Makes the Final Call?
Human oversight describes the degree to which a person can influence the final output of an AI system. Fabric identifies four distinct levels:
- Autonomous AI: In these systems, the AI’s output is the final decision, with no human intervention to change it. Interestingly, all autonomous AI systems in the Fabric repository were found to have the strictest form of institutional oversight, often due to high-risk applications.
- Conditionally Autonomous AI: Here, the AI can act on its own under certain conditions. However, if those conditions aren’t met (e.g., a confidence score is too low), a human steps in to make the ultimate decision.
- Human-Approved AI: For these systems, a human must explicitly approve or reject the AI’s output. If unsatisfied, the human might even modify the input data and re-run the system to get a different result.
- Human-Led with AI-Assistance: This is the most common pattern observed. The human always makes the final decision, using the AI’s output as a helpful tool or input. The AI assists, but the human retains full control over the final outcome.
Institutional Oversight Levels: The Organizational Guardrails
Institutional oversight refers to the broader organizational and external mechanisms that govern AI use. These range in strictness:
- Ad-Hoc Practice: The least strict, where governance decisions are made informally by individuals or teams.
- Organization Best Practice: Practices determined by the organization or its internal teams.
- Organization Policy: Formal internal rules that must be followed.
- Industry Standard: Standards set by industry bodies or official organizations.
- Regulation: The strictest level, where rules are legally enforceable, and non-compliance can lead to legal consequences. Many public sector AI systems, especially those handling sensitive data, fall under this category.
Also Read:
- Beyond Predictions: How Competing AI Narratives Shape Our Future and Governance
- Understanding and Governing AI Agent Behavior in the Digital Era
Key Takeaways and Future Directions
The study highlights that while practitioners often struggle to point to specific AI governance guidelines, they frequently adhere to existing regulations or domain-specific industry standards. This suggests a need for clearer, more integrated AI governance frameworks. The repository also reveals challenges with third-party AI systems, where organizations have less control and visibility into the system’s internal workings.
Fabric is designed to be an evolving tool. The researchers hope that by documenting these real-world examples, it will foster greater awareness among academics, industry professionals, and policymakers about how AI systems can be governed. The goal is to encourage continuous discussion about governance throughout the AI lifecycle, from pre-development to post-deployment, and eventually to assess the effectiveness of different governance mechanisms. You can learn more about this valuable resource by visiting the research paper directly: Documenting Deployment with Fabric: A Repository of Real-World AI Governance.
