TLDR: A new method called CLAWS uses attention patterns within LLMs to classify generated mathematical solutions as Typical, Creative, or Hallucinated without human evaluation. It analyzes how LLMs attend to different prompt sections (guideline, problem, reference solutions, instruction, response) and outperforms existing detection methods in accuracy and efficiency, offering a novel way to assess and improve LLM reasoning quality.
Large Language Models (LLMs) have made incredible strides in their ability to reason, especially when it comes to solving complex problems like those found in mathematics. These models, often trained with advanced techniques, can achieve impressive accuracy. However, while we often focus on how accurate an LLM’s answer is, there’s another crucial aspect of human intelligence that has been largely overlooked in LLM-generated solutions for reasoning tasks: creativity.
Traditionally, assessing creativity in LLMs has been confined mostly to writing tasks, using tests designed to measure creative writing. But when it comes to problem-solving, especially in fields like mathematics, evaluating creativity is much harder. This is due to two main challenges: first, it’s tough to precisely define what “creativity” means in a mathematical solution, and second, it usually requires human experts to make these judgments, which can be costly and difficult to scale.
To tackle these challenges, a new method called CLAWS (Creativity detection for LLM-generated solutions using Attention Window of Sections) has been proposed. CLAWS offers a novel way to categorize mathematical solutions generated by LLMs into three types: Typical, Creative, and Hallucinated. What makes CLAWS stand out is that it does this without needing human evaluators. Instead, it looks at how the LLM pays attention to different parts of the input prompt and its own output during the generation process.
The core idea behind CLAWS is that a solution’s creativity (or lack thereof, including hallucinations) is reflected in where the LLM focuses its attention. For instance, if an LLM is generating a “hallucinated” solution, it might be overly focused on the initial guidelines or problem statement, rather than the provided reference solutions or its own evolving response. Conversely, a truly creative solution might show a different pattern of attention, perhaps balancing attention across instructions and generating novel steps.
The input prompt given to the LLM is divided into five distinct sections for analysis: the Guideline (criteria for creativity), the Problem itself, Reference Solutions (examples of typical solutions), Instructions (to generate a novel solution), and the LLM’s own generated Response. CLAWS calculates the average attention weight for each of these sections. By normalizing these weights, it creates “section-wise attention ratios” which serve as features to classify the generated solution.
The researchers validated CLAWS using five different reasoning-focused LLMs (DeepSeek, Qwen, Mathstral, OpenMath2, and Oreal), each with 7-8 billion parameters. They tested it on a large dataset of 4,545 math problems collected from various contests. CLAWS was compared against five existing methods for detecting issues in LLM outputs, such as Perplexity and Logit Entropy. The results showed that CLAWS consistently outperformed these baselines in classifying solutions into the three categories.
Specifically, CLAWS demonstrated superior performance in metrics that give equal weight to all classes, indicating its effectiveness in identifying the more challenging “Creative” class, which often lies between “Typical” and “Hallucinated” solutions. It also proved highly effective in hallucination detection, even though the baseline methods were specifically designed for that task. Visualizations of the attention scores revealed clear distinctions: hallucinated solutions often showed a strong focus on the Guideline and Problem sections, while typical solutions leaned more towards reference solutions and instructions.
Beyond its accuracy, CLAWS is also remarkably efficient. It only requires extracting attention layers during the single response generation process and performs simple sum and average operations. This makes it faster than all the baseline methods, which often involve more complex computations or multiple generation steps. This efficiency, combined with its strong performance, makes CLAWS a promising tool for understanding and improving the quality and diversity of LLM-generated solutions.
Also Read:
- Beyond Final Answers: How RLVR Affects LLM Reasoning Traces
- Navigating the Learning Path: How Data Order Influences LLM Mathematical Reasoning
This research marks a significant step towards automatically assessing creativity in LLM-generated reasoning, moving beyond just accuracy. By defining creativity in a structured way and providing a method to detect it without human intervention, CLAWS opens new avenues for developing more sophisticated and human-like AI. You can find more details about this innovative work in the full research paper: CLAWS: Creativity Detection for LLM-Generated Solutions.
