TLDR: A new study reveals that single-run evaluations of Large Language Models (LLMs) are highly unreliable, with 83% of leaderboards showing rank inversions compared to multi-run aggregates. The research, conducted on the AI4Math Benchmark, demonstrates that averaging at least two runs significantly stabilizes model rankings and improves the robustness of evaluations. It recommends treating LLM evaluation as an experiment, reporting uncertainty, and using multiple repetitions, especially with stochastic decoding, to ensure trustworthy model comparisons and guide responsible AI deployment.
In the rapidly evolving world of Large Language Models (LLMs), accurate and reliable evaluation is paramount. These evaluations not only guide technological advancements but also build trust among users and stakeholders. However, a recent study delves into a critical, often overlooked aspect of LLM benchmarking: the impact of repetitions on the reliability of evaluation results.
The research, titled “DO REPETITIONS MATTER? STRENGTHENING RELIABILITY IN LLM EVALUATIONS,” by Miguel Angel Alvarado Gonzalez, Michelle Bruno Hernandez, Miguel Angel Peñaloza Perez, Bruno Lopez Orozco, Jesus Tadeo Cruz Soto, and Sandra Malagon, highlights a significant issue: LLM leaderboards frequently rely on single, stochastic runs. This practice, the authors argue, can lead to brittle and misleading conclusions.
The Problem with Single-Run Evaluations
The core finding of the study is striking: single-run leaderboards are highly unstable. The researchers re-evaluated eight state-of-the-art LLMs on the AI4Math Benchmark, performing three independent runs for each model and setting. Their analysis revealed that a staggering 83% of single-run evaluations showed at least one pairwise rank inversion when compared to the more robust three-run aggregate. This means that a model ranked higher in a single run might actually be lower when averaged across multiple trials, undermining the credibility of such comparisons.
While the statistical significance of pairwise differences remained robust (with a zero sign-flip rate), the actual rankings proved to be very sensitive to the inherent randomness of single runs, especially under stochastic decoding settings (temperature 0.7, top-p 0.95).
The Value of Repetitions
The study provides compelling evidence that repetitions are crucial for achieving stable and reliable LLM rankings. Averaging results across multiple runs, even just two, dramatically improves robustness. For instance, using two runs eliminated approximately 83% of the single-run rank inversions. While adding more runs (from one to three) only modestly reduced the standard error (around 5%), the primary benefit was in stabilizing the model rankings.
The researchers recommend treating LLM evaluation as an experiment, emphasizing the importance of reporting uncertainty and using at least two repetitions, particularly when employing stochastic decoding. This approach ensures that model comparisons are aligned with real-world reliability and are feasible even for smaller research teams.
Model Performance and Domain-Specific Insights
In terms of overall performance on the AI4Math benchmark, Grok-4 achieved the highest aggregate accuracy, closely followed by o3-mini and Claude Opus 4. Interestingly, Grok-4 and o3-mini were statistically indistinguishable, suggesting that even among top-tier models, clear superiority is not always guaranteed.
The domain-specific analysis offered further insights: Geometry and Probability were identified as the most challenging mathematical domains for all models, while Algebra proved to be the easiest. The study also highlighted complementary strengths between leading models: Grok-4 excelled with procedural persistence and formulaic calculations, whereas o3-mini showed advantages in spatial visualization problems. Both models, however, shared weaknesses in area-related and low-information optimization tasks, indicating that no single model dominates across all problem types.
Run-to-Run Consistency
The research also examined run-to-run instability, measuring how often a model’s answer changed across runs (flip-rate) and the consistency of model outputs (Intraclass Correlation Coefficient, ICC). Under the chosen stochastic decoding settings, GPT-4 family models changed their answers on about 35% of problems, while Grok-4 did so on approximately 10%. Grok-4 and o3-mini generally exhibited good reliability (ICC ≥ 0.77), indicating lower run-to-run noise, whereas GPT-4o-mini showed poor reliability (ICC ≤ 0.52).
Also Read:
- Unlocking Deeper Exploration in LLMs with Risk-Sensitive Reinforcement Learning
- Designing Algorithms: A Smarter Way for Language Models to Solve Graph Problems
Conclusion and Recommendations
The paper concludes that while LLM evaluations have progressed, the underreporting of statistical variation and the reliance on single runs remain significant limitations. Replication is essential for drawing reliable conclusions and stabilizing rankings, which are crucial for guiding technical progress and making informed deployment decisions.
The authors advocate for a shift in evaluation practices: researchers should establish an appropriate replication threshold tailored to their dataset and evaluation design, rather than prescribing a fixed number of runs. This approach ensures stable rankings without incurring unnecessary computational costs, contributing to more rigorous, transparent, and widely adoptable LLM evaluation practices. You can read the full research paper here.
