TLDR: DeepTRACE is a new audit framework that evaluates generative search engines and deep research LLM agents across eight measurable dimensions, including answer balance, factual support, and citation integrity. It uses statement-level analysis and matrices to assess how AI systems attribute evidence. Findings show that while these systems promise trustworthy synthesis, they often exhibit overconfidence, one-sidedness, and unsupported statements, with citation accuracy varying significantly. The framework highlights that more sources or longer answers do not guarantee reliability and emphasizes the need for improved AI system design for trustworthy information access.
In the rapidly evolving landscape of artificial intelligence, generative search engines and deep research LLM agents promise to deliver trustworthy, source-grounded information. However, users frequently encounter issues like overconfidence, weak sourcing, and confusing citation practices. To address these critical limitations, researchers have introduced DeepTRACE, a novel audit framework designed to track the reliability of these advanced AI systems.
Understanding DeepTRACE
DeepTRACE is a sociotechnically grounded framework that transforms previously identified failure cases into eight measurable dimensions. These dimensions span the answer text generated by the AI, the sources it uses, and its citation practices. The framework employs a statement-level analysis, breaking down AI-generated answers into individual statements and scoring their confidence. It also constructs detailed citation and factual-support matrices to thoroughly audit how these systems reason with and attribute evidence from start to finish.
The methodology behind DeepTRACE involves several key steps. First, it extracts the user query, the AI’s generated answer text, embedded citations, and the publicly accessible URLs of the sources. The answer text is then decomposed into individual statements, each assessed for its relevance to the user’s query and, for debate questions, whether it presents a pro, con, or neutral stance. An Answer Confidence score is also assigned to the overall response. Crucially, the full-text content of the listed sources is scraped, and two matrices are created: a Citation Matrix, indicating which statements cite which sources, and a Factual Support Matrix, showing which sources factually support each statement. These operations, while partly automated using LLM judges, are validated against human annotations to ensure reliability.
Key Metrics for Reliability
DeepTRACE evaluates AI systems across eight distinct metrics:
- One-Sided Answer: This metric checks if an answer to a debate question presents a balanced view, including both pro and con statements.
- Overconfident Answer: An answer is deemed overconfident if it is both one-sided and delivered with a high level of confidence.
- Relevant Statements: This measures the proportion of statements in the answer that directly address the user’s query, minimizing irrelevant ‘fluff’.
- Uncited Sources: This metric identifies sources listed by the AI that are not actually cited within the answer text.
- Unsupported Statements: It quantifies the fraction of relevant statements that are not factually supported by any of the listed sources.
- Source Necessity: This assesses whether the listed sources are truly necessary to support the statements in the answer, identifying potential redundancy.
- Citation Accuracy: This measures how accurately citations reflect that a source’s content supports the statement it is linked to.
- Citation Thoroughness: This metric evaluates the fraction of accurate citations included compared to all possible accurate citations based on the available evidence.
Also Read:
- Decoding AI Minds: How ‘Behavioral Fingerprinting’ Uncovers LLM Styles
- WebProber: AI Agents Enhance Web Usability Testing
Findings and Implications
The evaluation using DeepTRACE on popular public models like GPT-4.5/5, You.com, Perplexity, Copilot/Bing, and Gemini revealed significant insights. Generative search engines frequently produce one-sided, highly confident responses, especially for debate-style queries. They also often include a large fraction of statements unsupported by their own listed sources, and their citation practices can be uneven and inaccurate.
Deep-research configurations, while showing improvements in reducing overconfidence and achieving higher citation thoroughness, still exhibit high rates of one-sidedness on debate queries. Furthermore, a substantial portion of their statements remains unsupported, with citation accuracy ranging from 40% to 80% across different systems. A critical finding is that simply listing more sources or generating longer answers does not reliably translate into better grounding or accuracy; in fact, it can lead to ‘search fatigue’ for users and erode trust.
The research highlights that current generative search engines and deep research agents do not consistently deliver reliable outputs across all dimensions. While GSEs prioritize conciseness and relevance, they often sacrifice balance and factual support. Deep research agents, aiming for breadth and thoroughness, can become verbose, less relevant, and still struggle with unsupported claims. However, the study also points to systems like GPT-5(DR) demonstrating that near-ideal reliability across multiple dimensions is achievable with careful calibration.
DeepTRACE serves as a crucial tool for auditing these systems, moving beyond purely technical evaluations to consider their real-world impact on users. It underscores the need for substantial progress in AI-powered search to ensure safety, effectiveness, and to mitigate risks such as the formation of echo chambers and the erosion of user autonomy in information seeking. For more detailed information, you can read the full research paper here.
