TLDR: A recent evaluation by HKU Business School found that while Chinese large language models (LLMs) are improving, they still significantly lag behind top international counterparts like GPT-5 and Claude 4 Opus in controlling ‘hallucinations’—outputs that are factually incorrect or deviate from context. The study, which assessed 37 LLMs, highlighted that most models struggle more with factual inaccuracies than with maintaining contextual consistency.
Hong Kong, October 5, 2025 – The HKU Business School’s Artificial Intelligence Evaluation Laboratory has released a comprehensive ‘Large Language Model (LLM) Hallucination Control Capability Evaluation Report,’ revealing a notable performance gap between Chinese and leading international AI models. The report, initially released on September 9, 2025, underscores the persistent challenge of ‘hallucinations’ in LLMs, which produce plausible but factually incorrect or contextually inconsistent outputs.
Led by Professor Jack JIANG, Padma and Hari Harilela Professor in Strategic Information Management at HKU Business School, the research team conducted specialized assessments on 37 LLMs. This extensive evaluation included 20 general-purpose models, 15 reasoning models, and 2 unified systems. The study’s primary objective was to gauge how effectively these models prevent factual errors and maintain contextual consistency, a critical concern given the increasing deployment of LLMs in professional sectors such as knowledge services, intelligent navigation, and customer support.
Professor Jiang emphasized the importance of these findings, stating, ‘Hallucination control capability, as a core metric for evaluating the truthfulness and reliability of model outputs, directly impacts the credibility of LLMs in professional settings. This research provides clear direction for future model optimisation and advancing AI systems from simply being ‘capable of generating’ outputs to being more reliable.’
The evaluation categorized hallucinations into two types: factual and faithful. Factual hallucinations, often stemming from noisy training data, outdated knowledge, or confusion with complex facts, were assessed by comparing responses to factual sources. Faithful hallucinations, which arise from insufficient instruction parsing or a lack of strict output control, were evaluated based on the model’s accuracy in describing given information.
Key findings from the report indicate that international models continue to lead the pack. GPT-5 (Thinking) and GPT-5 (Auto) secured the top two positions, respectively, with the Claude 4 Opus series closely following. Among Chinese LLMs, ByteDance’s Doubao 1.5 Pro series emerged as the strongest performer. However, the report explicitly noted a ‘significant gap’ in overall capabilities between these leading Chinese models and their top-tier international counterparts.
Also Read:
- Dona Sarkar Advocates for Locally Developed AI Agents and Data in South Korea
- Navigating the AI Landscape: A Deep Dive into Generative and Applied AI Training in 2025
Overall, the study concluded that while most current large models demonstrate robust control over faithful hallucinations, they still encounter considerable difficulties in managing factual inaccuracies. This highlights an ongoing area for improvement in the development of more trustworthy and reliable AI systems.
