TLDR: Maryland is facing significant challenges with AI ‘hallucinations,’ particularly within its legal sector, where a recent case involved a lawyer using ChatGPT to generate fabricated legal content. This incident, alongside broader national trends of AI-generated inaccuracies in court documents, is prompting Maryland lawmakers to prepare comprehensive legislation for 2026 to establish guardrails for AI use across various applications, including legal, employment, housing, and education.
Maryland has officially joined the growing list of jurisdictions grappling with the perplexing phenomenon of AI ‘hallucinations,’ as evidenced by a recent legal proceeding that saw a lawyer utilizing generative artificial intelligence to produce fabricated case law. The case, identified as ‘Mezu v. Mezu’ in the California (CA) Maryland court on October 29, 2025, involved a lawyer who employed ChatGPT, leading to the submission of fabricated case law, false quotes, and misrepresented legal content. This resulted in a referral to the Attorney Grievance Commission, highlighting the severe professional repercussions of unverified AI use in legal practice.
This incident is not isolated but reflects a broader national trend. A comprehensive database tracking AI hallucination cases reveals that the United States alone has seen 341 such instances, with lawyers being responsible for 211 of these cases and pro se litigants for 283. The most common forms of AI-generated inaccuracies include fabricated case law (387 instances), false quotes (140 instances), and misrepresented legal content (218 instances).
In response to these emerging challenges, Maryland lawmakers are actively preparing for the 2026 legislative session to introduce measures aimed at guiding the responsible and strategic deployment of artificial intelligence. Discussions are underway to establish necessary ‘guardrails’ to mitigate the risks associated with AI’s potential for bias and misinformation. Delegate Jesse Pippy of Frederick County emphasized the urgency, stating, “There needs to be guardrails on AI — similar to what is happening in the legal community, where AI has spit out inaccurate case law, where lawyers have gotten in trouble using AI-generated memos that have given inaccurate information.”
Also Read:
- FDA Committee to Deliberate on Risks and Regulation of AI Mental Health Chatbots
- Governments Emphasize Urgent Need for Robust AI Governance Frameworks Amidst Rapid Advancements
A dedicated workgroup was established earlier this year to formulate recommendations for protecting consumers from AI-related harms, particularly in critical areas such as employment and housing. Furthermore, lawmakers are exploring how AI can be leveraged beneficially in educational settings, for instance, to assist educators in lesson planning and support students struggling in core subjects. The upcoming legislative session is expected to see several bills return in updated forms, including proposals to study AI use in schools and to regulate AI-driven hiring tools, as Maryland seeks to balance innovation with public safety and ethical considerations in the rapidly evolving AI landscape.
