TLDR: A position paper argues that NeurIPS, the premier AI conference, should take on the crucial role of catalyzing scientific consensus on AI policy. It highlights a current void in consensus-forming mechanisms, explains why NeurIPS is uniquely suited for this task by drawing parallels with the IPCC’s success in climate science, and proposes concrete pilot initiatives like working groups, dedicated tracks, and debates. The paper also provides examples of policy areas, such as AI model evaluation and regulatory threshold design, where scientific agreement is urgently needed to inform effective AI governance.
Designing effective policies for artificial intelligence (AI) is a significant challenge facing society today. To create wise AI policies, decision-makers need to rely on strong evidence and a shared understanding among scientists. While there are ways to generate and gather evidence, a clear process for forming scientific consensus on AI policy has been missing. A new position paper argues that NeurIPS, a leading conference in AI, should step up to fill this gap and actively help build scientific agreement on AI policy.
The paper, authored by Rishi Bommasani from the Stanford Institute for Human-Centered Artificial Intelligence (HAI), highlights that while governments worldwide acknowledge the importance of good AI policy, they often differ on the specifics. The policy decisions made today will shape AI’s future and its impact on humanity. Many AI researchers are now publishing work on AI policy, emphasizing the need for rigorous evidence and scientific consensus.
Rigorous evidence refers to information gathered through transparent, replicable, and sound methods. Scientific consensus is defined as a general agreement among experts in a field, built on accumulated evidence over time. While evidence generation is robust, with thousands of submissions to conferences like NeurIPS, critical information about AI’s impacts often remains unclear. Evidence synthesis, which involves extracting key insights from vast amounts of data, is also gaining traction, notably with initiatives like the International Scientific Report on AI led by Yoshua Bengio.
However, the paper points out a significant void in mechanisms for forming scientific consensus on AI policy. It argues that NeurIPS is uniquely positioned to lead this effort due to its strengths. NeurIPS is considered the premier global AI gathering, attracting top scientists and serving as a beacon for scientific leadership. Its ability to convene the AI community and its strong reputation make it an ideal platform for both internal organization and external communication of scientific consensus.
The paper also examines alternatives and explains why they are less suitable. The International Scientific Report on AI, while excellent for evidence synthesis, is not designed for ongoing consensus formation. The high-profile AI summit series (like the UK AI Safety Summit or the AI Seoul Summit) are often influenced by geopolitics and lack the scientific depth needed for meaningful consensus. Other AI conferences, while valuable, don’t match NeurIPS’s broad attendance and reputation across various AI sub-areas.
To guide NeurIPS in this new role, the paper draws lessons from the Intergovernmental Panel on Climate Change (IPCC), which has successfully built scientific consensus on climate policy for decades. The IPCC’s process emphasizes legitimacy through inclusive expert selection and reasoned agreement, credibility through government integration while maintaining scientific independence, and impact through policy-relevant but not prescriptive reports, including a ‘Summary for Policymakers’ that governments edit with scientific veto power.
Based on the IPCC model, the paper proposes several pilot initiatives for NeurIPS. These include creating a standing working group of community leaders and AI policy researchers to provide ongoing leadership and dialogue throughout the year. Another idea is a dedicated track at the conference to promote scholarship focused on forging scientific consensus, such as surveys of expert views or meta-analyses of conflicting evidence. Finally, NeurIPS could host special sessions for spirited debates on areas lacking consensus, supplemented by pre- and post-conference surveys to gauge community agreement on specific topics.
The paper offers concrete examples of policy areas where scientific consensus is urgently needed. One such area is evaluation selection. Policymakers often require AI models to undergo evaluations to ensure safety or compliance, but there’s no clear scientific consensus on what constitutes ‘state-of-the-art’ evaluation methods. Agreement on the validity, reliability, and cost of evaluations would be invaluable for policy design.
Another critical area is threshold design. Policies often use quantitative thresholds, like the amount of computing power used to train an AI model, to determine which entities are subject to regulation. However, there’s ongoing debate among experts about the effectiveness and appropriateness of such thresholds as proxies for risk. Scientific consensus could help identify viable metrics, explore hybrid solutions, and develop principles for making decisions despite trade-offs between predictive validity and measurement costs.
The authors acknowledge potential counterarguments, such as the view that policy engagement is not NeurIPS’s role or that AI researchers disagree too much for consensus. They respond by stating that NeurIPS already plays a role in scientific consensus through peer review and that AI policy topics are increasingly within its scope. They also argue that even with disagreement, consensus can be found on common ground, such as better measurement infrastructure, or on procedures to reduce uncertainty.
Also Read:
- Navigating the AI Policy Maze: A New Framework for Computer Science Education
- The Sandbox Configurator: A Framework for Standardized AI Assessment in Regulatory Environments
In conclusion, the paper makes a compelling case for NeurIPS to embrace a leadership role in fostering scientific consensus on AI policy. By doing so, NeurIPS can significantly contribute to the development of evidence-based AI policies that ensure AI produces beneficial outcomes for society. You can read the full paper for more details on this important proposal: NeurIPS Should Lead Scientific Consensus on AI Policy.
