TLDR: Achieving accountable artificial intelligence in government hinges on reforming and scrutinizing public procurement processes, particularly addressing how AI systems are acquired through non-traditional pathways like donations or small-dollar purchases, which often bypass standard oversight. New research highlights the varied and often insufficient current practices, advocating for a focus on procurement as a primary lever for responsible AI governance.
The pursuit of accountable artificial intelligence within government operations must fundamentally begin with the procurement process, according to a recent analysis published by Tech Policy Press on July 15, 2025. The article, authored by Nari Johnson, Elise Silva, and Hoda Heidari, underscores that while much attention is paid to the technology itself, the methods by which AI systems are acquired by public entities are often overlooked, creating significant loopholes for oversight.
A pivotal example cited is the 2018 case of the New Orleans Police Department, which utilized predictive policing software donated by Palantir. This arrangement, because it involved no exchange of money, circumvented the city’s standard procurement regulations, including requirements for city council debate and approval. Consequently, key city officials and council members were unaware of the partnership, raising immediate concerns from civil rights groups regarding potential racial bias and lack of transparency.
Inspired by such incidents, a research team comprising scholars from Carnegie Mellon University and the University of Pittsburgh investigated AI purchasing processes in the public sector. Through interviews with nineteen city employees across seven anonymous U.S. cities, their findings revealed a wide disparity in procurement practices among localities. This variability significantly impacts the feasibility of effective AI governance.
The research emphasizes that procurement serves as a critical lever for governments to uphold public values such as safety, non-discrimination, privacy, and accountability, especially in the absence of comprehensive federal AI regulation. However, current efforts to reform AI procurement often focus solely on the conventional competitive solicitation process, which involves Requests for Proposals (RFPs), vendor scoring, and contract negotiations.
Crucially, the study highlights that many AI systems bypass this formal solicitation process entirely. Alternative acquisition pathways include small-dollar purchases made with government-issued purchasing cards, AI donated by companies, systems acquired through university partnerships, or even freely available public tools like ChatGPT. Furthermore, vendors are increasingly integrating new AI features into existing contracts without public or city staff notification. This means that many existing resources designed for responsible AI procurement are not applicable to the majority of current AI acquisitions.
The article points out that while competitive solicitations offer benefits for responsible AI governance, many city employees perceive them as inefficient. This leads to a reliance on alternative pathways, posing a key question: how can local governments establish consistent AI governance norms when most tools are acquired outside formal processes? The answer, the authors suggest, lies in understanding who is involved in each type of acquisition.
Oversight capacity varies significantly based on how local governments organize their procurements. Some cities have centralized oversight, with IT staff vetting all software acquisitions for quality and risk. Others operate with decentralized IT governance, granting individual departments autonomy. The research suggests that a one-size-fits-all reform approach is unlikely to succeed, advocating for either training individual departments in AI risk assessment or reconfiguring workflows to establish centralized reviews for ethical standards.
The past year has seen growing momentum among local governments, supported by organizations like the Government AI Coalition, to integrate responsible AI considerations into procurement. The research, presented at the 2025 ACM Conference on Fairness, Accountability, and Transparency, adds a vital layer by detailing how AI procurement actually functions in practice. It raises critical questions for local governments: how to establish oversight for bypassed proposals, identify responsible parties for risk management, and restructure workflows for meaningful AI solution evaluation.
Also Read:
- U.S. Government Accelerates AI Integration Across Critical Sectors, Raising Efficiency and Safety Debates
- Switzerland Forges Unique Path in AI Regulation, Opting for Pluralistic Approach Over Comprehensive Act
In conclusion, the article asserts that public procurement is where some of the most consequential decisions about public sector AI are made. To truly understand why an AI system is adopted and whose interests it serves, the focus must shift to the initial acquisition process.
