TLDR: OPENESTIMATE is a new benchmark evaluating LLMs on numerical estimation under uncertainty using real-world data from public datasets. It requires models to provide probabilistic predictions (Bayesian priors) and assesses their accuracy and calibration against statistical baselines. Findings show LLMs are often inaccurate and overconfident, performing similarly to estimates from a few data samples, though their priors can improve posteriors when combined with data. Elicitation methods impact calibration by adjusting model uncertainty.
In today’s rapidly evolving technological landscape, large language models (LLMs) are increasingly being deployed in critical real-world domains such as healthcare, finance, and various forms of knowledge work. These applications often require LLMs to navigate situations with incomplete information and make decisions under uncertainty. However, a significant challenge exists: most current evaluations for LLMs focus on problems with clear-cut answers and well-defined success criteria, leaving a gap in understanding how these models perform when faced with real-world ambiguity.
Addressing this crucial gap, researchers from MIT CSAIL — Alana Renda, Jillian Ross, Michael Cafarella, and Jacob Andreas — have introduced a new benchmark called OPENESTIMATE. This innovative framework is designed to rigorously evaluate LLMs on numerical estimation tasks that demand the synthesis of substantial background information and the expression of predictions as probabilistic priors. Unlike traditional benchmarks, OPENESTIMATE focuses on scenarios where the “right answer” isn’t easily found in training data, thus testing true reasoning skills rather than memorization.
What is OPENESTIMATE?
OPENESTIMATE is an extensible, multi-domain benchmark that uses real-world data to create challenging estimation tasks. The core idea is to generate “derived conditional random variables” from large-scale, publicly available observational datasets. These datasets span diverse fields, including labor economics (Glassdoor), private markets (Pitchbook), and human health (NHANES).
The benchmark constructs variables that are unlikely to be explicitly documented in pretraining corpora. These can be simple marginal statistics, like “the mean salary of data scientists,” or more complex conditional statistics, such as “the average funding raised by non-tech companies outside the US with more than 10 people.” By systematically varying conditioning attributes, OPENESTIMATE creates a vast array of fine-grained estimation targets that are grounded in real-world data but empirically difficult for LLMs to predict accurately.
How are LLMs Evaluated?
Instead of asking for simple point estimates, OPENESTIMATE requires LLMs to express their predictions as full probability distributions, specifically as Gaussian or Beta distributions, depending on whether the target variable is continuous or a proportion. This approach allows for a more nuanced evaluation, distinguishing between predictions that are accurate by chance and those that reflect genuine understanding and calibrated uncertainty.
The evaluation focuses on two key dimensions:
- Accuracy: This measures how closely the mean of the LLM’s predicted distribution aligns with the true, empirical ground-truth value. Performance is quantified using Mean Absolute Error (MAE) and a “win rate” against statistical baselines derived from small empirical samples. An error ratio below one indicates the LLM is more accurate than a small, noisy sample.
- Calibration: This assesses whether the model’s stated uncertainty matches empirical frequencies. A well-calibrated model should have its ground-truth values fall into predicted quartiles with the correct long-run frequency. The Expected Calibration Error (ECE) quantifies deviations from this ideal.
Key Findings from the Evaluation
The researchers evaluated six frontier LLMs, including models from Meta Llama, OpenAI GPT, and Qwen. The findings revealed that LLM-elicited priors are often inaccurate and overconfident. In many cases, these models performed no better, and sometimes worse, than estimates derived from just a handful of samples from the underlying population. No single model family consistently outperformed the others across all domains, though larger reasoning models generally showed better comparative performance.
Interestingly, while standalone LLM priors were often inaccurate, they could be effectively combined with data. When LLM priors were updated with a small number of samples, the resulting posteriors often outperformed baselines that used uninformative priors, especially when data availability was limited.
The study also explored how different inference-time settings affect prediction quality. While factors like temperature (reasoning effort) and system prompts had negligible impact on accuracy, the “elicitation protocol” (how uncertainty is requested from the model) significantly influenced calibration. Direct elicitation worked best for reasoning models, while quantile elicitation proved superior for non-reasoning models. This improvement in calibration was primarily attributed to an increase in the models’ expressed uncertainty, counteracting their natural tendency towards overconfidence.
Also Read:
- Unlocking Individual Thought: A New Benchmark for Language Models
- JudgeSQL: Enhancing Text-to-SQL Accuracy Through Intelligent Selection
Implications for Future LLM Development
The OPENESTIMATE benchmark highlights that despite their impressive capabilities, frontier LLMs still face significant challenges in probabilistic estimation and reasoning under uncertainty, especially when dealing with real-world, open-ended data. The benchmark offers a robust platform for developing models that are better at probabilistic estimation and more accurately reflect their uncertainty. This research is crucial for advancing LLMs towards more reliable and trustworthy deployment in sensitive domains. You can find the full research paper here: OPENESTIMATE: Evaluating LLMs on Reasoning Under Uncertainty with Real-World Data.
