TLDR: California’s Department of Forestry and Fire Protection (Cal Fire) has launched an AI chatbot aimed at providing wildfire information, but a recent investigation by CalMatters revealed significant shortcomings. The chatbot struggles to provide accurate or up-to-date details on wildfire containment, evacuation orders, and emergency supply lists, raising concerns about the reliability of AI tools in critical public safety applications.
SACRAMENTO, CA – The California Department of Forestry and Fire Protection (Cal Fire) has introduced an artificial intelligence chatbot designed to offer Californians ‘critical fire prevention resources and near-real-time emergency information.’ However, a recent investigation by CalMatters, published on July 9, 2025, has cast a shadow on the tool’s effectiveness, revealing its inability to consistently provide accurate and timely information, particularly concerning wildfire containment and evacuation orders.
The chatbot, which Cal Fire states was developed independently of Governor Gavin Newsom’s 2023 executive order to integrate AI across state government services, was touted by the governor’s office in May as a step towards improving public access to vital information. Despite its stated purpose, the CalMatters probe found several critical failures.
One significant issue highlighted is the chatbot’s inaccuracy in describing wildfire containment. For instance, when queried about the Ranch Fire in San Bernardino County, a 4,293-acre blaze, the chatbot reported it as 50% contained based on a June 10 update. However, at the time of the query, the fire was 85% contained, making the chatbot’s information six days out of date. This demonstrates a critical lag in real-time data crucial during active emergencies.
Furthermore, the investigation revealed that the chatbot struggles with providing essential evacuation information. It cannot reliably inform users about active evacuation orders and, when it does attempt to address the topic, it inconsistently identifies law enforcement as the issuing authority. Similarly, obtaining a list of evacuation supplies proved difficult; the chatbot only provided the information when users asked the precise phrase, ‘What should I have in my evacuation kit?’ Slight variations in wording often led to unhelpful responses.
Experts are weighing in on the findings. Daniel Ho, a law professor at Stanford University specializing in government AI use, emphasized the necessity of thorough evaluation. ‘Evaluation is not an afterthought,’ Ho stated. ‘It should be part of the standard expectation when we pilot and roll out a system like this.’ Mila Gascó-Hernandez, research director at the University at Albany’s Center for Technology in Government, echoed these concerns, stressing the importance of reliability in emergency situations: ‘If a fire is coming and you need to know how to react to it, you do need both accuracy and consistency in the answer.’
The chatbot, built by South Carolina-based company Citibot, which specializes in AI-powered chatbots for government agencies, is intended to be hosted by Cal Fire until at least 2027. While the chatbot performed accurately when asked about active wildfires and basic agency information, its inconsistencies with nuanced or time-sensitive queries raise questions about its readiness for critical public safety roles.
Also Read:
- EPA’s New IT Chief Cautions Against Overreliance on AI as a Universal Solution
- European Media Alliance Unveils AI Chatbot ‘ChatEurope’ to Combat Disinformation
Issac Sanchez, Cal Fire’s Deputy Chief of Communications, acknowledged the chatbot’s shortcomings. He stated that improvements are currently underway and urged the public to be patient as the agency works to refine the tool’s responses. This incident underscores the broader challenges California faces as it rapidly integrates AI into public services, balancing the potential benefits of efficiency with the paramount need for accuracy and reliability in emergency communications.
