TLDR: A study comparing ChatGPT’s ability to extract legal principles (PoLs) from Italian court judgments against regular expressions (Regex) found that ChatGPT struggles with accuracy, quantity, and generates convincing but fabricated information (hallucinations). While ChatGPT could generate the Regex, the Regex itself, when applied manually, significantly outperformed ChatGPT’s direct extraction, suggesting that current AI lacks true reasoning and integration capabilities in complex legal tasks.
In an era where artificial intelligence (AI) is increasingly integrated into various professional domains, a recent study delves into the performance of ChatGPT within the intricate field of law. The research, titled “ChatGPT Unveils Its Limits: Principles of Law Deliver Checkmate,” explores ChatGPT’s efficacy in extracting “Principles of Law” (PoLs) from legal documents, comparing its capabilities not just against human experts but also against a baseline using regular expressions (Regex).
The study highlights a significant limitation of ChatGPT: while it possesses the necessary knowledge, it often struggles to synthesize, reason through, and integrate this information to produce comprehensive and accurate results. This suggests that genuine intelligence, particularly in nuanced legal tasks, remains a uniquely human attribute.
Legal professionals, including judges and lawyers, frequently encounter challenges related to limited IT support and technical proficiency. AI-driven chatbots, designed for user-friendliness and contextual understanding, offer a promising solution. ChatGPT, known for its ability to generate human-like responses with precise grammar and adaptability, appears well-suited for smaller legal entities with constrained technology budgets. However, the paper emphasizes the critical need for empirical validation of these theoretical capabilities.
The core task examined was the extraction of PoLs, which are fundamental frameworks for interpreting and applying legal rules, evolving from specific case facts into broader, general statements that ensure legal certainty and predictability. Automating this extraction process is crucial for enhancing efficiency in legal research and case resolution.
Researchers used a dataset of 60 Italian judgments concerning LGBTQIA+ rights, randomly selected from the De Jure database. These cases were particularly relevant due to the frequent absence of clear legislative guidance, making PoLs indispensable. A team of three legal domain experts manually annotated these judgments, categorizing PoLs as direct (quoted), indirect (paraphrased), or implicit (lacking a clear prior citation). This meticulous human annotation served as the gold standard for comparison, identifying a total of 682 PoLs.
The human annotators themselves faced challenges, noting variations in judgment styles, diverse quotation mark usages, different paraphrasing techniques for indirect and implicit PoLs, and inconsistent citation formats. They also observed that quoted passages could contain legislative references or narrative excerpts, not exclusively PoLs.
When ChatGPT was put to the test, the researchers employed a “Few-shot learning” approach, providing the AI with detailed prompts and examples. Despite these efforts, ChatGPT’s performance revealed several shortcomings. It was most effective when extracting a small number of PoLs (2-4) but its accuracy declined significantly as the number of PoLs increased. Regardless of the judgment’s length, ChatGPT consistently extracted only an average of 3 to 5 PoLs. Explicit direct PoLs, with their clear structure, were the easiest for the AI to identify.
A critical issue identified was the occurrence of “hallucinations” – instances where ChatGPT generated entirely fabricated PoLs that were unrelated to the source text. These hallucinations were particularly problematic because they often appeared highly convincing, mimicking the structure and topic of real PoLs, making them difficult to detect even for legal experts without a careful comparison to the original text. The study also noted ChatGPT’s lack of determinism, where the same query could yield different results, necessitating frequent chat restarts to mitigate errors.
In a pivotal part of the study, the researchers explored an alternative: using ChatGPT to generate a regular expression (Regex) for PoL extraction. This ChatGPT-generated Regex was then applied to the dataset using a simple text editor. Interestingly, ChatGPT itself admitted it could not directly apply the Regex to the provided files, citing potential encoding or protection issues.
The Regex-based extraction, however, significantly outperformed ChatGPT’s direct extraction. While it still missed some PoLs and occasionally identified non-PoL passages (like legislative references or narratives), it did not suffer from hallucinations. These “Not-PoLs” were easily identifiable and discardable by human experts, unlike ChatGPT’s fabricated content. The Regex method detected an average of 6 PoLs per judgment, more than double ChatGPT’s average.
The comprehensive comparison underscored the superior accuracy of human annotators. However, the Regex, despite its simplicity, proved to be a more reliable automated tool than ChatGPT for this specific task. The study’s findings suggest that while ChatGPT possesses vast knowledge, it lacks the inherent ability to plan, break down complex tasks, and integrate information effectively – hallmarks of true intelligence. The presence of convincing hallucinations further diminishes its utility as a reliable support tool in critical legal contexts.
Also Read:
- Unmasking Uncertainty: How “Label Indeterminacy” Challenges AI in Law
- Assessing AI Judges: A New Benchmark for Web Development Quality
The authors propose the methodology of using a ChatGPT-generated Regex, applied via a text editor, as a practical pipeline for legal experts without specialized IT skills to accelerate PoL identification. This approach is user-friendly, replicable, and accessible. Future research will broaden the dataset to include diverse legal topics, evaluate other large language models (LLMs) like Deepseek and Lama, and experiment with structured prompting techniques to potentially reduce hallucinations and improve accuracy. For more details, you can read the full research paper here.
