TLDR: A new study challenges the idea that more reasoning always improves Large Language Models’ confidence calibration. It found that increasing an LLM’s “thinking budget” actually leads to systematic overconfidence and worse accuracy in assessing expert confidence. Conversely, giving LLMs access to external search tools dramatically boosts their accuracy, suggesting that information access, not deeper reasoning, is the key to better confidence in knowledge-intensive tasks.
Large Language Models (LLMs) are becoming indispensable tools for answering complex questions and managing workflows. However, a critical challenge remains: ensuring these models are accurately calibrated in their confidence, especially to avoid being “confidently wrong.” A recent research paper, titled “Don’t Think Twice! Over-Reasoning Impairs Confidence Calibration,” by Romain Lacombe, Kerrie Wu, and Eddie Dilworth, delves into this very issue, offering surprising insights that challenge conventional wisdom in AI development.
The study systematically evaluates how an LLM’s reasoning capabilities and the computational budget allocated for “thinking” affect its ability to accurately assess confidence. The researchers used the CLIMATE X dataset, which contains climate-related statements meticulously labeled with human expert confidence levels, and expanded their analysis to include human and planetary health domains.
The Paradox of Over-Reasoning
One of the paper’s most striking findings directly contradicts the popular “test-time scaling” paradigm, which suggests that giving LLMs more computational resources for longer, more intricate chains of thought should improve performance. The study found that while initial, modest increases in reasoning budget (e.g., 64-192 tokens) did offer slight improvements in accuracy, pushing the reasoning budget further consistently impaired rather than improved confidence calibration. Extended reasoning led to systematic overconfidence that worsened with longer thinking budgets, yielding diminishing and even negative returns beyond a certain point.
For instance, the Gemini 2.5 Flash model showed an initial accuracy bump from 41.7% to 45.0% on the IPCC dataset with a modest thinking budget. However, beyond 256 tokens, accuracy collapsed, bottoming out near 35.7% at 768 tokens. This suggests that longer chains of thought might introduce spurious rationales or circular reasoning, ultimately harming both accuracy and calibration. Furthermore, models exhibited increasing and marked overconfidence as the reasoning budget grew, often overestimating certainty in inherently uncertain categories.
The Power of Information Access
In stark contrast to the negative effects of over-reasoning, the study found that search-augmented generation dramatically outperformed pure reasoning. By retrieving relevant external evidence, LLMs achieved an impressive 89.3% accuracy, compared to 48.7% for reasoning-only models. This significant improvement highlights a crucial point: information access, rather than the depth of internal reasoning or the inference budget, appears to be the critical bottleneck for improving confidence calibration in knowledge-intensive tasks.
When models were given the ability to perform an online search to contextualize each statement, the task was nearly perfectly solved. This suggests that LLM understanding of complex scientific claims is, to a surprising extent, limited by access to the right evidence. Once the retrieval component provides salient passages, the model can assign the correct categorical confidence almost every time.
Also Read:
- Calibrating Large Language Models with a Structured Play Framework
- Beyond Surface-Level Self-Reports: A Deeper Look at AI Introspection
Implications for AI Development
These findings have profound implications for how we design and deploy LLMs, especially in high-stakes domains like climate science and public health, where accurate confidence assessment is paramount. The paper suggests that simply scaling up reasoning compute or “thinking budget” may yield diminishing returns. Instead, practitioners should prioritize investments in improving information retrieval and evidence synthesis capabilities.
The research also utilized the CLIMATE X dataset, comprising 8,094 expert-annotated sentences from IPCC Assessment Reports, and a novel dataset from the International Agency for Research on Cancer (IARC) for carcinogenicity classifications. This allowed for a robust evaluation across different critical domains.
In conclusion, the study “Don’t Think Twice! Over-Reasoning Impairs Confidence Calibration” (available at https://arxiv.org/pdf/2508.15050) offers a cautionary message for the “test-time scaling” paradigm. It underscores that for LLMs to be truly reliable and well-calibrated, especially in complex, knowledge-intensive fields, providing them with robust access to external, verifiable information is far more effective than merely encouraging them to “think longer.”
