TLDR: A new benchmark, the Self-Execution Benchmark, evaluates large language models’ (LLMs) ability to predict aspects of their own responses, such as associations, refusal to answer, or question difficulty. The study found that current LLMs, including larger and more capable models, generally perform poorly on these tasks, indicating a fundamental limitation in their self-awareness and ability to reason about their own behavior, a capability not typically measured by standard benchmarks.
Large language models (LLMs) have shown remarkable capabilities in various tasks, from generating text to solving complex problems. However, a recent research paper introduces a novel perspective on evaluating these models: their ability to predict aspects of their own responses. Unlike a Turing machine, which can simulate its own behavior, LLMs lack the inherent ability to ‘self-execute’ or internally run their own code to see what their output would be. Instead, they must rely on internal estimations of their own behavior.
This fundamental limitation raises a crucial question: how well can an LLM anticipate its own responses? To address this, researchers Elon Ezra, Ariel Weizman, and Amos Azaria from Ariel University, Israel, developed the Self-Execution Benchmark. This benchmark is designed to measure a model’s capacity to predict properties of its output, such as whether a question will be difficult for it, if it will refuse to answer, or what kinds of associations it might produce.
The Benchmark’s Experiments
The Self-Execution Benchmark comprises three distinct experiments, each probing a different facet of an LLM’s self-prediction capabilities:
1. The Association Test: This experiment evaluates a model’s ‘associative awareness.’ Models were first asked to generate five associative words for a target word. Later, in a separate session, they were presented with the target word and a list of option words, and asked to predict which of those options appeared in their previously generated list. The results were not encouraging; even the best-performing model, o4-mini, achieved only 63% accuracy, barely better than random guessing (50%). Models tended to over-predict the inclusion of words, leading to high recall but low precision.
2. The Restriction Recognition Test: This test assesses an LLM’s awareness of its own content restrictions. Models were given questions, some legitimate and some prohibited (e.g., involving criminal activity). They first predicted whether they would answer the question or refuse. Then, in a separate session, they were prompted to answer the question. Finally, they were asked to confirm if their response actually provided an answer. Performance here was also low, with Gemini 2.5 Pro scoring only 32%. A significant finding was the ‘over-niceness rate,’ indicating that models often provided answers to questions they believed they should have refused, demonstrating excessive compliance rather than accurate self-restriction.
3. The Difficulty Assessment Test: This experiment examined a model’s ability to rank the difficulty of questions for itself, based on expected correctness and the number of tokens used. Models were given groups of four questions from various topics and asked to sort them from easiest to hardest. While larger models (like Claude, DeepSeek, GPT, and Gemini) performed better than smaller ones (around 70% vs. 60%), none of the models performed exceptionally well compared to the 50% random baseline. Surprisingly, models with explicit reasoning capabilities did not show a clear advantage over their non-reasoning counterparts in this task, despite performing better on the individual questions themselves.
Key Findings and Implications
Across all experiments, the study consistently found that current LLMs struggle significantly with self-prediction. Performance did not reliably improve with increased model size or sophistication, suggesting a persistent ‘blind spot’ in how these models represent and reason about their own behavior. The average performance on the restriction recognition test was the highest, followed by the difficulty assessment test, while the association test showed the lowest scores, often close to random chance.
The researchers also compared the self-execution benchmark scores with those from common LLM benchmarks (like MMLU, MATH-500, HumanEval). They found a low correlation (0.59) between general model ability and self-execution scores. This suggests that the Self-Execution Benchmark measures distinct capabilities not captured by traditional evaluations, highlighting a unique aspect of LLM intelligence.
Also Read:
- Assessing Language Models as Moral Guides: A New Benchmark for Ethical Reasoning
- Unpacking LLM Strategic Reasoning: A New Benchmark Approach
Future Directions
The paper discusses the challenges of evaluating human performance on such a benchmark, as humans don’t operate in discrete ‘sessions’ or lack memory like LLMs. It also proposes new, LLM-specific benchmarks, such as a ‘token-injection recovery’ test, where random words are inserted into an LLM’s ongoing response to see if it can recover and still produce the correct answer. Future work includes expanding the benchmark, exploring fine-tuning techniques to improve self-prediction, and investigating how LLMs might behave if given tools for actual self-execution, albeit with necessary safeguards to prevent issues like infinite loops.
In conclusion, despite their impressive external performance, current LLMs lack meaningful self-executive awareness. They are not yet capable of accurately predicting or reasoning about their own behavior, revealing a deeper misalignment between their beliefs about their limitations and their actual capabilities. This research opens new avenues for understanding and improving the internal workings of large language models.
