TLDR: A new research paper introduces Distractor-Normalized Coherence (DINCO), a method to improve the calibration of verbalized confidence in large language models (LLMs). DINCO addresses LLM overconfidence and ‘suggestibility’—the tendency to accept claims when uncertain—by having the model generate and evaluate alternative, contradictory claims. By normalizing confidence across these self-generated distractors and integrating self-consistency, DINCO provides more accurate, less saturated, and ultimately more trustworthy confidence estimates for LLMs across various tasks and models.
Large language models (LLMs) have become incredibly powerful, demonstrating impressive knowledge across many domains. However, a significant challenge remains: their tendency to be overconfident, often reporting high certainty even when their answers are incorrect. This miscalibration can severely undermine user trust and raise safety concerns, especially when these models are used for critical decision-making.
A recent research paper, “Calibrating Verbalized Confidence with Self-Generated Distractors”, introduces a novel approach called Distractor-Normalized Coherence (DINCO) to address this problem. The authors, Victor Wang and Elias Stengel-Eskin from The University of Texas at Austin, propose a method that helps LLMs provide more accurate and reliable confidence estimates.
The Problem of Overconfidence and Suggestibility
The paper highlights that LLMs often express their confidence in human-like ways, such as stating a percentage or confirming an answer’s correctness. However, these verbalized confidence scores are frequently miscalibrated. The researchers hypothesize that this overconfidence often stems from what they call ‘suggestibility’. This means that when an LLM has little information about a claim, it might be more prone to accept it simply because it was presented in the context. Their empirical findings validate this, showing higher suggestibility on claims where the model’s accuracy is lower.
Another issue identified is ‘confidence saturation’. This occurs when the model’s reported confidence scores cluster into a few high bins, making them uninformative. For example, if an LLM always says it’s 99% confident, that score loses its meaning, even if it’s sometimes correct. This saturation makes it difficult to distinguish between truly certain and uncertain responses.
Introducing DINCO: A New Approach to Calibration
DINCO tackles these issues by estimating and accounting for an LLM’s suggestibility bias. The core idea is to have the model verbalize its confidence not just on the main claim, but also on several ‘self-generated distractors’ – alternative, incompatible claims. By normalizing the main claim’s confidence against the total verbalized confidence across these distractors, DINCO gains a more nuanced understanding of the model’s true certainty.
Imagine asking an LLM, “When was Kang Ji-hwan born?” and it confidently says “1980.” If you then ask it, “Was Kang Ji-hwan born in 1990?” and it also expresses high confidence, this inconsistency reveals a lack of true knowledge. DINCO leverages this incoherence. If an LLM is highly confident in multiple contradictory statements, its initial high confidence in any single statement should be discounted.
How DINCO Works in Detail
The method involves several key steps:
- Distractor Generation: The LLM generates several plausible but inaccurate alternative claims (distractors) related to the original question or statement. For instance, if the main claim is “Kang Ji-hwan is highly acclaimed,” distractors might be “Kang Ji-hwan is widely ridiculed” or “Kang Ji-hwan is notoriously disliked.”
- Independent Confidence Verbalization: The LLM then independently states its confidence for each of these distractors, as well as the original claim.
- Normalization and Weighting: The confidence in the main claim is then normalized by the total confidence across all distractors. To ensure accuracy, an off-the-shelf Natural Language Inference (NLI) model is used to downweight distractors that are redundant or do not truly contradict the main claim. This prevents overcounting in the normalization factor.
- Integrating Self-Consistency: DINCO also incorporates ‘self-consistency’, a popular method that samples multiple generations from an LLM and measures their agreement. By combining this ‘coherence within generation’ with the ‘coherence across validations’ (from distractors), DINCO provides a more robust confidence estimate. This addresses the ‘generator-validator disagreement’ where an LLM might generate an answer but then inconsistently validate it.
Also Read:
- A Cost-Effective Way to Measure LLM Intelligence: The Consistency Score
- Unlocking More Reliable AI Reasoning: A Solvability-Based Approach to Multiple-Choice Questions
Key Findings and Impact
The research demonstrates that DINCO significantly improves calibration across various models (open-source like Qwen3-8B and closed-source like GPT-4.1) and tasks, including short-form question answering (TriviaQA, SimpleQA) and long-form biography generation (FActScore).
Notably, DINCO provides less saturated confidence estimates, meaning the scores are more spread out and informative, as opposed to always being clustered at very high confidence. The paper shows that DINCO, even with fewer inference calls (e.g., 10 calls), outperforms self-consistency methods that use significantly more samples (e.g., 100 calls). This highlights DINCO’s efficiency and the unique value of its approach.
By offering more calibrated and usable confidence scores, DINCO represents a crucial step towards making LLMs more trustworthy and reliable for human users and agentic systems. This method helps LLMs better understand what they truly know and express that uncertainty in a meaningful way, fostering greater confidence in AI outputs.
