TLDR: A new framework called MAD-Fact uses a multi-agent debate system and a fact importance hierarchy to accurately evaluate the factual consistency of long-form text generated by Large Language Models (LLMs). It introduces a Chinese long-form dataset, LongHalluQA, and shows that this debate-based approach, which weighs facts by importance, outperforms single-model evaluations and correlates well with human judgment, especially for Chinese content.
The widespread adoption of Large Language Models (LLMs) has brought about remarkable advancements in text generation, but it has also amplified concerns regarding the factual accuracy of their outputs. This issue is particularly pressing in high-stakes domains such as biomedicine, law, and education, where the generation of inaccurate or biased content, often referred to as hallucination, can lead to serious consequences.
Traditional methods for evaluating factual accuracy typically focus on short texts, like question-answering tasks. However, real-world applications increasingly demand LLMs to generate long-form content, sometimes spanning hundreds or thousands of words. Unlike short texts, long-form content often incorporates multiple perspectives and complex logical structures, making its factual evaluation significantly more challenging. Existing short-text evaluation methods are often inadequate for these complex scenarios, highlighting the need for more nuanced evaluation mechanisms.
Addressing Key Challenges in Factuality Evaluation
Current research on long-form factuality evaluation has made strides, particularly with approaches that decompose text into atomic claims for knowledge-base verification. However, several critical challenges persist:
- Scarcity of Chinese Long-Form Benchmarks: Most existing long-form factuality benchmarks are designed for English, lacking comprehensive resources for Chinese content. This oversight hinders the objective evaluation of LLM performance in Chinese generation tasks, as culturally specific entities, historical events, and linguistic nuances are often overlooked.
- Biases in Single-Model Evaluation Frameworks: Many evaluation frameworks rely on a single model for factual verification. However, even advanced LLMs can hallucinate, leading to incorrect or inconsistent judgments. This makes single-model architectures prone to systematic biases, potentially misrepresenting the true factuality of generated content.
- Neglect of Fact Importance in Metrics: Existing metrics often treat all factual claims equally, regardless of their relative importance. For instance, a central claim might hold the same weight as an auxiliary detail, even though their impact on user relevance or task outcomes can differ significantly. This uniform treatment can undermine the ability of evaluation methods to capture the true factual quality of long-form outputs, especially in critical applications.
Introducing MAD-Fact and LongHalluQA
To overcome these limitations, researchers have proposed a systematic approach that integrates large-scale long-form datasets, multi-agent verification mechanisms, and weighted evaluation metrics. This work introduces LongHalluQA, a new Chinese long-form factuality dataset, and develops MAD-Fact, a debate-based multi-agent verification system.
LongHalluQA is a comprehensive benchmark for Chinese long-form factuality evaluation, comprising 2,746 high-quality samples across seven topics, including Chinese culture, natural sciences, and social sciences. This dataset addresses the critical need for resources to evaluate Chinese long-form generation.
MAD-Fact is a Multi-Agent Debate system designed for factual verification. It aims to mitigate single-model biases and improve reasoning reliability through structured interactions among different agent modules. The system consists of three types of agents:
- The Clerk Agent: This agent is responsible for decomposing the LLM-generated long-form response into multiple atomic claims, filtering out unverifiable content like subjective statements.
- The Jury: Composed of multiple Evaluator Agents, each assuming a distinct professional role (e.g., Public, Critic, News Author, Scientist, Psychologist, Data Analyst), the Jury assesses the factuality of each atomic claim through external retrieval and multi-agent debate. These agents engage in multi-round discussions, reviewing shared knowledge, examining previous statements, and deciding whether to use external search tools to supplement their knowledge.
- The Judge Agent: This agent aggregates the individual evaluations and explanations from the Jury for each atomic claim, determining a final factual prediction (TRUE or FALSE) based on a majority voting principle. It then calculates an overall factuality score for the entire response.
A key innovation within MAD-Fact is the introduction of a fact importance hierarchy model. This model quantifies the relative significance of individual claims by analyzing their frequency of occurrence across multiple expert-generated reference answers. Claims mentioned more frequently by experts are assigned greater factual importance weights. This allows for weighted evaluation metrics, such as Weighted Precision, Weighted Recall, and Weighted F1-Score, which better reflect the true factual quality of generated content by accounting for the varying importance of facts.
Also Read:
- Co-Sight: A Framework for Trustworthy and Efficient AI Agent Reasoning
- Making AI More Reliable: A Framework for Fact-Checking Language Models
Experimental Findings and Impact
Experiments on various fact-checking datasets, including the newly constructed LongHalluQA, demonstrated MAD-Fact’s superior performance compared to strong baselines like SAFE and FIRE. The multi-agent debate framework proved effective in enhancing fact-checking capabilities. The study also explored different debate rules and found that strategies balancing external retrieval with internal knowledge utilization yielded the best results.
Benchmarking nine mainstream LLMs on LongFact (an English dataset) and LongHalluQA revealed several insights: larger LLMs generally maintain higher factual consistency, and domestic models excel on Chinese content. For instance, on LongHalluQA, Chinese models like QwQ-32B, Doubao-1.5-Pro, and DeepSeek-V3 significantly outperformed international counterparts like GPT-4-Turbo, highlighting potential cultural biases and imbalances in multilingual capabilities of general-purpose models.
Crucially, the proposed weighted evaluation metrics, guided by the fact importance hierarchy, showed a strong correlation with human judgments (Pearson correlation coefficient r=0.701, p=0.036). This validates that the system can effectively assess the factual quality of long-text responses in a way that aligns with human perception.
This research provides a structured and reliable framework for evaluating and enhancing factual reliability in long-form LLM outputs. It offers practical guidance for model selection and optimization, paving the way for the safe and effective deployment of LLMs in sensitive domains. For a deeper dive into the methodology and results, you can access the full research paper here.
