TLDR: The paper introduces ESI, a novel method to measure epistemic uncertainty in Large Language Models (LLMs). ESI assesses an LLM’s reliability by observing how much its output changes when the input prompt is subtly altered in a way that preserves the original meaning (semantic-preserving intervention). This technique helps identify when an LLM relies on superficial correlations rather than genuine understanding. Experiments show ESI is more effective and computationally efficient than existing methods, improving LLM reliability by better predicting when their answers might be incorrect.
Large Language Models (LLMs) have become an integral part of our daily lives, transforming from experimental technology into widely used tools. However, a significant challenge persists: LLMs frequently generate incorrect or untruthful content, a phenomenon commonly known as hallucination. This issue severely impacts their reliability and limits their application in critical domains.
To address this, researchers are exploring Uncertainty Quantification (UQ), a promising approach to improve model trustworthiness. Uncertainty is generally categorized into two types: aleatoric uncertainty, which stems from inherent randomness in the data (e.g., multiple plausible answers to a question), and epistemic uncertainty, which arises from the model’s lack of knowledge about the underlying data-generating process. Epistemic uncertainty is often considered a more reliable indicator of a model’s trustworthiness.
Quantifying uncertainty in LLMs, especially for free-form text generation, is not straightforward. The vast and complex nature of natural language outputs makes it difficult to accurately measure uncertainty. Existing methods often rely on sampling multiple outputs and measuring their semantic variation, which can be computationally expensive and typically estimates total uncertainty rather than specifically epistemic uncertainty.
A Novel Approach: Epistemic Uncertainty Quantification via Semantic-Preserving Intervention (ESI)
A new research paper introduces a novel method called ESI, which stands for Epistemic Uncertainty Quantification via Semantic-preserving Intervention. This approach connects the uncertainty of LLMs to their stability under semantic-preserving interventions from a causal perspective. The core idea is that a truly reliable LLM, one that understands the underlying causal mechanisms of language, should produce stable outputs even when its input prompt is altered in a way that doesn’t change its core meaning.
Imagine a human answering a question. If they truly understand the topic, their answer won’t change significantly if the question is rephrased slightly, as long as the meaning remains the same. However, if they are just guessing based on superficial cues, a slight rephrasing might lead to a different answer. ESI applies this principle to LLMs.
How ESI Works
The ESI method measures the variation in an LLM’s output before and after a “semantic-preserving intervention” is applied to the input prompt. This intervention subtly changes the prompt’s surface form without altering its underlying semantic meaning. The paper proposes two main intervention techniques:
-
Paraphrasing (Para): This involves using another LLM to generate semantically equivalent rephrased versions of the original prompt. A good paraphrase changes the linguistic structure while keeping the meaning intact.
-
Skip-One-Char (SOC): A simpler, more computationally efficient method where one random character is removed from the latter portion of randomly selected words in the prompt. This creates a minor textual change that typically doesn’t affect the prompt’s meaning.
After applying an intervention, ESI quantifies the average shift in the token predictive distribution of the *same* response. This means it looks at how the model’s confidence in generating each word of the original answer changes when the subtly altered prompt is fed in. A larger shift indicates higher epistemic uncertainty.
The researchers provide theoretical justification that ESI effectively estimates epistemic uncertainty, distinguishing it from aleatoric uncertainty. Unlike methods that try to reconstruct the entire output space, ESI focuses on the stability of a single response, making it more stable and efficient.
Practical Advantages and Performance
In practice, ESI uses the Hellinger distance to measure the difference between probability distributions, which is a stable and well-behaved metric. It also focuses on the top-k most probable tokens to improve efficiency, making it potentially applicable even to closed-source models that provide access to top-k log probabilities.
Extensive experiments were conducted across various LLMs (Llama2-chat7B, Mistral-Nemo-Instruct12B, Llama3-Instruct8B, Llama3-Instruct70B, Llama3.1-Instruct8B, Qwen2.5-Instruct14B, Qwen3-Instruct4B) and a variety of question-answering datasets, including those with single ground-truth answers (SciQ, TriviaQA) and those with high aleatoric uncertainty or multiple correct answers (AmbigQA, TruthfulQA, CoQA). The results consistently showed that ESI outperforms state-of-the-art methods in predicting the correctness of LLM-generated answers, as measured by AUROC (Area Under the Receiver Operating Characteristic curve).
ESI demonstrated particular strength in datasets with high aleatoric uncertainty (like AmbigQA and TruthfulQA) and open-book datasets (like CoQA). This is because ESI specifically targets epistemic uncertainty, avoiding misattributing inherent data uncertainty to model errors. The method also proved to be significantly more computationally efficient, being 3-5 times faster than baselines and achieving good performance with fewer samples.
Ablation studies confirmed the critical importance of semantic preservation in the intervention process. If an intervention alters the meaning of the prompt, ESI’s effectiveness drops significantly, reinforcing the core assumption of the method.
Also Read:
- Decoding LLM Confidence: The Signal in Reasoning Trace Length
- Enhancing Trust in Large Language Models with SAFER’s Two-Stage Risk Control
Conclusion
The ESI method offers a fresh perspective on Uncertainty Quantification for LLMs by leveraging the principle of causal invariance. By measuring how stable an LLM’s output is under meaning-preserving input changes, ESI provides an effective and efficient estimate of epistemic uncertainty. This advancement can significantly enhance the reliability of LLMs, helping to identify when their outputs can be trusted and when they might be prone to hallucination. For more details, you can read the full research paper here.
