TLDR: California’s Department of Forestry and Fire Protection (Cal Fire) launched an AI chatbot, ‘Ask CAL FIRE,’ to provide critical fire prevention resources and real-time emergency information. However, recent reports by CalMatters and The Markup reveal significant shortcomings, including inaccurate wildfire containment data, inconsistent responses to evacuation supply queries, and an inability to provide real-time evacuation orders. Experts emphasize the need for thorough evaluation and public involvement before deploying such AI tools in critical public services.
California’s ambitious push into artificial intelligence, spearheaded by Governor Gavin Newsom’s 2023 executive order to enhance government efficiency, has hit a snag with the rollout of a new AI chatbot by the California Department of Forestry and Fire Protection (Cal Fire). The chatbot, named ‘Ask CAL FIRE,’ was designed to offer Californians streamlined access to vital fire prevention resources and near-real-time emergency information. However, investigative reporting by CalMatters and The Markup, published on July 9, 2025, has brought to light critical deficiencies in the chatbot’s performance.
The primary concerns revolve around the chatbot’s accuracy and consistency. CalMatters found that ‘Ask CAL FIRE’ struggles to accurately describe the containment status of wildfires. For instance, when queried about the Ranch Fire in San Bernardino County, the chatbot reported it as 50% contained based on a June 10 update, while the actual containment was 85% six days later. This six-day lag in information could have serious implications during fast-moving wildfire events.
Furthermore, the chatbot’s ability to provide reliable information on evacuation supplies is inconsistent. While a direct question like ‘What should I have in my evacuation kit?’ yielded a specific list, variations such as ‘go bag,’ ‘wildfire ready kit,’ or ‘fire preparedness kit’ often resulted in a generic prompt to visit Cal Fire’s ‘Ready for Wildfire’ website or a message stating, ‘I’m not sure about the specific items you should have.’ This lack of semantic understanding raises questions about its utility in high-stress situations.
Perhaps the most critical failing is the chatbot’s inability to provide information about active evacuation orders. When asked who issues such orders, the bot sometimes correctly identified law enforcement, but at other times responded with ‘I don’t know.’ Issac Sanchez, Cal Fire’s Deputy Chief of Communications, acknowledged that it is ‘reasonable to expect the chatbot to be able to answer questions about evacuations,’ suggesting that if no orders exist, the response should clearly state, ‘there doesn’t appear to be any evacuations associated with this incident.’
The ‘Ask CAL FIRE’ chatbot, which offers resources in 70 languages, was developed by Citibot, a South Carolina-based company specializing in AI-powered chatbots for government agencies. Cal Fire intends to host the tool until at least 2027. According to Newsom’s office, the chatbot is part of a broader initiative to modernize public communication tools.
However, experts are raising alarms about the deployment process. Daniel Ho, a law professor at Stanford University specializing in government AI use, emphasized that ‘Evaluation is not an afterthought. It should be part of the standard expectation when we pilot and roll out a system like this.’ Mila Gascó-Hernandez, research director for the University at Albany’s Center for Technology in Government, echoed this sentiment, highlighting the importance of both accuracy and consistency, especially when citizens need critical information during emergencies. She stressed that users ‘are not going to think about ‘what’s the nice way to ask the chatbot?”
Sanchez stated that his team of approximately four people tested the chatbot before its launch by submitting anticipated public questions. He also noted that Cal Fire is actively working on improvements by analyzing user queries to refine the bot’s responses. ‘The big message we want to get across,’ Sanchez said, ‘is be patient.’
Also Read:
- California Tax Agency Integrates Generative AI to Boost Call Center Efficiency
- New “AI Darwin Awards” Launched to Spotlight Major Artificial Intelligence Failures of 2025
However, experts argue that such extensive testing and refinement should occur before public release. Ho suggested establishing clear performance criteria and benchmarks with an independent third party before vendor selection. Gascó-Hernandez added that public involvement in piloting the system to ensure it meets citizen needs is crucial for effective AI deployment in public services. The challenges faced by ‘Ask CAL FIRE’ underscore the complex balance between leveraging AI for public good and ensuring its reliability and safety.
