TLDR: This research paper examines how three dominant narratives—AI existential risk proponents, accelerationists, and critical AI scholars—shape governance decisions by presenting distinct, often deterministic, visions of AI’s future. It analyzes their differing views on societal impact, technology’s role, and human agency. The paper argues that these narratives, which are already influencing global policies, limit democratic imagination and proposes a pragmatic governance framework that focuses on observable outcomes, iterative refinement, and broad public participation to navigate AI’s complex and uncertain trajectory.
Artificial intelligence (AI) is rapidly transforming our world, but how we talk about its risks profoundly shapes how we decide to govern it. A recent research paper, “The Stories We Govern By: AI, Risk, and the Power of Imaginaries,” delves into how different ways of imagining AI’s future influence policy decisions and regulatory approaches. The authors, Ninell Oldenburg and Gleb Papyshev, explore three major narrative groups that dominate discussions around AI risk.
The paper introduces the concept of “sociotechnical imaginaries,” which are collective visions of how technology fits into society and the kind of future we prioritize. These imaginaries aren’t just abstract ideas; they actively influence institutional agendas, funding, and regulatory paths by embedding specific views on risk, responsibility, and progress into governance structures.
Three Dominant Narratives of AI Risk
The researchers analyze three distinct groups, each with a unique perspective on AI risk:
1. AI Existential Risk (X-risk) Proponents: This group, represented by organizations like the Machine Intelligence Research Institute (MIRI), warns of an apocalypse driven by artificial general intelligence (AGI). They believe that once AI surpasses human intelligence, it will pursue its own goals, potentially leading to catastrophic outcomes for humanity. Their vision emphasizes a future where AI is a disempowering force and a direct threat to human survival. They advocate for strict global governance and regulatory frameworks, including a metaphorical “off switch,” to prevent AI from spiraling out of control. For them, safety and survival are paramount, and they see the current research community as not taking these risks seriously enough.
2. Accelerationists: Figures like Marc Andreessen represent this group, which envisions a radically optimistic future fueled by relentless technological advancement. They see AI as a transformative force that will solve global challenges like inequality and climate change, leading to a post-scarcity society. For accelerationists, technology is the only viable path forward, and any regulation is seen as a threat to innovation and geopolitical advantage. They believe in individualistic, market-driven progress, where “builders” (entrepreneurs and technologists) are the primary agents of societal improvement. Critics of technology are often dismissed as irrational or outdated.
3. Critical AI Scholars: This perspective, exemplified by the Distributed AI Research Institute (DAIR), rejects grand futurist narratives. Instead, they focus on the present-day harms of AI, such as surveillance, labor exploitation, and racial bias. They argue that the most urgent risks are already here and are deeply intertwined with existing power structures and inequalities. DAIR envisions a future where technology serves marginalized communities, and harmful systems are refused, even if technically feasible. Their approach is rooted in collective agency, mutual care, and epistemic justice, prioritizing community leadership and equitable resource distribution over market demands or technical possibilities.
Divergent Views on Key Dimensions
The paper highlights how these three imaginaries differ across four key dimensions:
- Normative Vision of the Future: MIRI fears collapse, Andreessen champions acceleration and abundance, while DAIR calls for restorative and reparative futures grounded in justice.
- Social Order and Values: MIRI focuses on human survival against powerful AI, Andreessen promotes individualistic, market-driven progress, and DAIR advocates for collective agency, mutual care, and dismantling unjust hierarchies.
- Role of Science and Technology: MIRI sees AI as a tool for catastrophic innovation, Andreessen views it as an inherently redemptive force, and DAIR frames technology as a contested, political terrain that must be governed ethically and collectively.
- Determinism vs. Agency: All three narratives, despite their differences, exhibit a form of determinism. MIRI suggests catastrophe is inevitable without drastic global intervention. Andreessen believes in the inevitable positive progress driven by free markets. DAIR argues that AI will inevitably perpetuate existing inequalities unless there’s radical, community-led reorientation. Each group believes their prescribed intervention is the singular path to redirecting AI’s future.
Implications for AI Policy
The research points out that these deterministic visions are already shaping global AI policies. For instance, the EU AI Act’s risk-based taxonomy, the UK AI Security Institute’s focus on “frontier” models, and the US Executive Order on AI’s compute-based thresholds all reflect assumptions about predictable risks and trajectories. Similarly, national AI strategies in Brazil, Russia, and China embed their own forms of developmentalist, geopolitical, or strategic determinism.
Also Read:
- The Imperative for a Unified Global AI Governance Framework
- Unpacking the ‘Regulation Sacrifice’: A Critical Look at AI Governance
A Path Forward: Pragmatic AI Governance
The authors argue against these deterministic approaches, proposing a “pragmatic governance framework” for AI. This approach emphasizes assessing ideas based on their practical consequences rather than abstract ideologies. It calls for focusing on observable outcomes, systematically cataloging and responding to actual harms, and creating clear mechanisms for reparative actions. Pragmatic governance rejects rigid, binary thinking, acknowledging that technological trajectories are uncertain and multifaceted. It promotes flexibility, experimentation, and continuous refinement of policies based on real-world evidence. Furthermore, it prioritizes broad public participation and inclusive deliberation to define and redefine the public good as AI systems evolve.
While acknowledging potential challenges like a perceived lack of normative anchoring or risk of policy paralysis, the paper asserts that pragmatic governance can integrate ethical concerns through evidence-based, participatory processes and build decision-making capacity through iterative learning. It also suggests that this approach can better manage long-term risks by treating speculative scenarios as hypotheses to be tested and contextualized within broader empirical knowledge, rather than relying solely on worst-case predictions.
Ultimately, the paper advocates for moving beyond fixed, predetermined visions of AI’s future. By embracing uncertainty and focusing on practical, observable impacts, a pragmatic approach can foster robust, adaptable, and democratically legitimate AI governance that genuinely serves societal needs and values. You can read the full paper here.
