TLDR: Researchers propose a new method to continuously monitor large-scale Generative AI models for reliability issues like hallucinations and semantic drift. They compare a rule-based “deterministic Knowledge Graph” with an “LLM-generated Knowledge Graph” derived from real-time news. By tracking structural metrics (ICR, IPR, CI) and a hallucination score, their automated framework detects deviations and anomalies, providing an interpretable and scalable way to ensure AI trustworthiness without relying on static benchmarks.
Generative AI (Gen AI) models have brought about a revolution in various fields, but they also come with significant challenges, including issues like hallucinations (making up facts), semantic drift (changing meaning over time), and inherent biases. These models often operate like ‘black boxes,’ making it difficult to understand how they arrive at their outputs and to evaluate them transparently and objectively.
Current methods for evaluating Gen AI largely depend on human assessment, which isn’t practical for large-scale, continuous monitoring. These methods are often static, meaning they don’t adapt as AI models constantly evolve, and they can be compromised if evaluation data accidentally becomes part of the model’s training data, leading to an overestimation of performance.
To tackle these problems, a new research paper titled “Continuous Monitoring of Large-Scale Generative AI via Deterministic Knowledge Graph Structures” introduces an innovative approach. The authors, Kishor Datta Gupta, Mohd Ariful Haque, Hasmot Ali, Marufa Kamal, Syed Bahauddin Alam, and Mohammad Ashiqur Rahman, propose a systematic way to continuously monitor and evaluate Gen AI reliability using Knowledge Graphs (KGs).
The core of their methodology involves building two parallel KGs:
Deterministic Knowledge Graph (KG)
This KG is constructed using explicit, rule-based methods. It relies on predefined ontologies (structured representations of knowledge), domain-specific dictionaries, and clear rules for extracting entities (like people, places, organizations) and their relationships. This deterministic KG serves as a stable, explainable baseline or a ‘pseudo-ground truth’ against which the Gen AI’s output can be compared.
Also Read:
- Navigating the Ethical Landscape of Generative AI: A New Evaluation Framework
- Building Flexible AI: How Models Learn, Forget, and Adapt
LLM-Generated Knowledge Graph (KG)
This KG is dynamically created by a Large Language Model (LLM) itself. It processes real-time textual data streams, such as live news articles, to extract entities and relationships. Using live news ensures that the data is authentic, helps mitigate biases from repetitive training, and prevents adaptive LLMs from simply memorizing predefined benchmarks.
The innovation lies in continuously comparing these two KGs. By measuring the structural differences and semantic discrepancies between the deterministic (rule-based) KG and the LLM-generated KG, the system can identify when the Gen AI model deviates from established facts or rules. The researchers employ several established KG metrics to quantify these differences:
-
Instantiated Class Ratio (ICR): This metric shows how many of the defined categories (classes) in the knowledge system are actually used to create specific items (entities) in the KG. A high ICR suggests the model is effectively utilizing its knowledge structure.
-
Instantiated Property Ratio (IPR): This measures the proportion of relationships (properties) that are actually used between entities. A high IPR indicates a rich and expressive knowledge graph.
-
Class Instantiation (CI): This metric evaluates how entities are distributed across the hierarchy of classes, ensuring a balanced and semantically rich use of the ontology.
In addition to these structural metrics, the framework includes a Hallucination Score. This score is calculated by validating extracted entities against the original input source and the predefined KG schema. Any entity that is not present in the input or violates the schema is flagged as a hallucination, providing a more objective way to detect factual inaccuracies.
An automated, real-time monitoring framework continuously computes the deviations between the two KGs. It uses dynamic anomaly thresholds, which adjust based on historical data, to proactively identify and flag significant deviations. This means the system can promptly detect semantic anomalies or hallucinations as they occur, providing early warnings for potential issues.
The methodology is divided into three phases: first, constructing both KGs from real-time news; second, evaluating and comparing their structural integrity using the defined metrics; and third, implementing an adaptive, continuous evaluation framework with anomaly detection.
The researchers conducted an empirical study comparing nine different LLMs across three time periods. They found that models like Gemini 1.5, Vicuna, and Qwen consistently showed better alignment with ground truth values for ICR and CI, indicating a more faithful use of schema classes. Qwen also demonstrated a richer utilization of relational predicates (IPR). Conversely, models like GPT-3.5-turbo and Llama 3.3 often showed significant underutilization of the schema. While hallucination rates varied, some models exhibited very low rates in certain contexts, while others occasionally introduced spurious information.
This framework offers substantial benefits. It provides a continuous, vendor-agnostic, and interpretable way to monitor LLM behavior, moving beyond the limitations of static benchmarks. It doesn’t require access to the model’s internal workings or labeled data, relying instead on explainable structural signals. This makes it valuable for individuals tracking personal LLMs, enterprises evaluating models for procurement, product teams during experimentation, and even in regulated industries for compliance reporting. The focus is on detecting persistent deviations over time, offering early warnings for regressions and unstable updates.
A demo website showcasing this monitoring pipeline is currently live at monitorllm.com. This research represents a crucial step towards ensuring the reliability and trustworthiness of large-scale generative AI in real-world applications. For more in-depth information, you can refer to the full research paper available at arxiv.org/pdf/2509.03857.
