TLDR: This research introduces the “Scaling Law by Difficulty,” showing that too much easy training data hurts LLM reasoning, while high-difficulty data improves it. To address this, they propose the Structured Solution Template (SST) framework, a three-stage method that uses explicit solution templates and a difficulty-aware curriculum to teach LLMs procedural reasoning. SST significantly boosts accuracy and efficiency on complex math tasks like GSM8K and AIME24, demonstrating that structured, difficulty-aware training is key for advanced LLM problem-solving.
The paper “Can Structured Templates Facilitate LLMs in Tackling Harder Tasks? : An Exploration of Scaling Laws by Difficulty” by Zhichao Yang and his colleagues introduces a significant finding in how Large Language Models (LLMs) learn and perform on complex tasks, especially in mathematics. The researchers uncovered a “Scaling Law by Difficulty,” which challenges conventional wisdom about training data.
Traditionally, it might be assumed that more training data always leads to better performance. However, this study reveals a U-shaped curve in model performance related to training data complexity. This means that providing LLMs with an excessive amount of low-difficulty data can actually hinder their ability to abstract and reason effectively. Instead of learning true problem-solving skills, the models tend to memorize patterns, which makes them less reliable when faced with new, complex problems. Conversely, training with high-difficulty data significantly enhances their reasoning capabilities.
Motivated by this crucial insight, the researchers developed a new framework called the Structured Solution Template (SST). This framework is designed to explicitly teach LLMs procedural reasoning by using solution templates and a curriculum that varies in difficulty. The SST framework aims to overcome the limitations highlighted by the Scaling Law by Difficulty.
The Three Stages of the SST Framework
The SST framework operates in three main stages. The first stage involves fine-tuning LLMs using structured solution-template chains. During this process, a dynamically weighted loss mechanism is applied to prioritize the procedural logic within these templates. This ensures the model focuses on the steps of reasoning rather than just the surface text.
In the second stage, solution templates are injected at prompt-time. These templates act as cognitive scaffolds, guiding the LLM’s inference process in real-time. A lightweight, LoRA-based generator creates problem-specific chain prompts on the fly, enhancing reasoning and efficiency without altering the main model’s weights. For example, given a geometry problem, the generator might provide a chain like “Square each leg → Sum the squares → Take the square root” to guide the model.
The third stage integrates these approaches into a comprehensive curriculum fine-tuning process. Here, the model is explicitly taught to self-plan, execute, and self-correct. This involves identifying the most challenging errors from the initial training, adding chain templates and self-reflection notes using a powerful LLM like DeepSeek-R1, and then using Group Relative Policy Optimization (GRPO) to refine the model’s ability to plan, solve, and correct problems autonomously. This stage focuses on building robust “plan-then-execute” reasoning capabilities directly into the model.
Also Read:
- Unlocking LLM Potential: A New Approach to Prompt Engineering with Smaller Models
- Bridging Language and Logic: How AI Models Tackle Complex Optimization Problems
Empirical Gains and Further Analysis
The effectiveness of the SST framework was rigorously tested across seven different benchmarks, including well-known datasets like GSM8K, AIME24, and a new “Dynamic En” benchmark designed for cross-domain reasoning. The results were impressive, showing that SST significantly improves both accuracy and efficiency, particularly on harder problems. For instance, it achieved a notable gain of up to 6.2 points on GSM8K and a 2.2 point increase on AIME24. These empirical gains confirm that template-guided, difficulty-aware training is vital for advanced LLM reasoning.
The study also provides further analysis, breaking down the contributions of each stage. Stage 1, with its chain-weighted supervised fine-tuning, establishes strong procedural priors and performs well on difficult, curated problems. Stage 2, with prompt-time chain injection, excels on problems requiring real-time scaffolding, such as AIME24 and AIME25. Stage 3, by integrating both mechanisms, consistently outperforms or matches the best of the individual stages across all tasks, demonstrating the complementary benefits of weighted template learning and dynamic chain prompting.
Furthermore, the research explored the impact of prompt-time chain injection on efficiency. For simpler tasks like GSM8K, injecting chains led to a significant reduction in token count (over 60%), suggesting the model follows the chain more directly. For harder tasks like AIME24 and AIME25, accuracy improved with only modest changes in token count, indicating that the chains serve as a scaffold for more active, deliberative reasoning.
Ablation studies confirmed the importance of structured chain annotation, weighted loss, and the role of GRPO in curriculum fine-tuning. Emphasizing abstract chain tokens during training was crucial to prevent overfitting to shallow heuristics and to leverage template annotation effectively. The use of LoRA for the chain generator also showed benefits, particularly for more challenging cross-domain tasks.
In conclusion, this research highlights a critical aspect of LLM training: the quality and difficulty of training data are more important than sheer quantity. The Structured Solution Template framework offers a promising approach to teaching LLMs genuine procedural reasoning, enabling them to tackle complex problems with greater accuracy and efficiency. For more in-depth information, you can refer to the full research paper here.
