TLDR: Reasoning Curriculum is a two-stage training method for LLMs. It first develops core reasoning skills in math through a cold start and math-only reinforcement learning (RL) with verifiable rewards. Then, it transfers and refines these skills across diverse domains (STEM, code, logic, etc.) using joint RL. This approach consistently improves general reasoning performance in models like Qwen3-4B and Llama-3.1-8B, demonstrating that math-first elicitation enhances transferable cognitive behaviors.
Large Language Models (LLMs) have shown remarkable capabilities in reasoning, especially in specialized areas like mathematics and coding. However, extending these strong reasoning skills across a broader range of domains has been a persistent challenge for open-source research. A new paper introduces an innovative approach called “Reasoning Curriculum” to address this very issue, aiming to bootstrap and broaden LLM reasoning from a math-first foundation.
The core idea behind Reasoning Curriculum is a simple yet effective two-stage training process. The researchers observed that math is particularly conducive to eliciting reasoning skills through reinforcement learning (RL), even with weak supervision. They hypothesized that math can act as a powerful driver for discovering fundamental reasoning abilities that can then be transferred to other domains.
Stage 1: Math-Focused Skill Elicitation
The first stage of the curriculum focuses on developing core reasoning skills primarily within the math domain. It begins with a brief “cold start,” where the LLM undergoes supervised fine-tuning on a small set of math problems. This initial step exposes the model to rich thought processes and diverse reasoning patterns. Following this, the model engages in math-only Reinforcement Learning (RL) using verifiable rewards. This RL phase is crucial for strengthening and consolidating the cognitive skills necessary for complex problem-solving, such as subgoal setting, enumeration, backtracking, and verification. These skills, while present in LLMs, are often weakly expressed and need targeted elicitation.
Stage 2: Cross-Domain Skill Transfer and Refinement
Once the LLM has developed robust reasoning skills in math, the second stage aims to transfer and refine these abilities across a wider array of domains. This is achieved through “joint RL” on a mixed-domain corpus. This corpus includes data from various fields such as STEM, code, simulation, logic, and tabular tasks, alongside math problems. By training jointly on this diverse data, the model learns to adapt and apply its newly acquired reasoning capabilities to different contexts. The beauty of this curriculum is its minimalism; it doesn’t require specialized reward models beyond standard verifiability checks, making it backbone-agnostic and easy to adopt.
Also Read:
- FunReason-MT: Enhancing AI’s Ability to Use Tools in Complex Conversations
- A New Way for Large Language Models to Self-Verify Their Reasoning
Evaluation and Key Findings
The Reasoning Curriculum was rigorously evaluated on two popular LLMs: Qwen3-4B and Llama-3.1-8B. The results were consistently positive. For Qwen3-4B, the model trained with Reasoning Curriculum (RC-Qwen) not only outperformed similarly sized baselines but also proved competitive with, and sometimes even surpassed, much larger 32B systems across various benchmarks. For Llama-3.1-8B, after minor adjustments like initializing from an instruct model and introducing a difficulty curriculum within the Math-RL stage, the approach again yielded significant performance improvements across all domains.
A detailed analysis of cognitive skill usage revealed that the Reasoning Curriculum significantly increased the frequency of advanced cognitive behaviors like verification and backtracking. This supports the hypothesis that math-first training enhances transferable reasoning skills. Ablation studies further confirmed that both the cold-start and the math-RL stages are essential components, each contributing meaningfully to the overall performance gains.
In essence, the Reasoning Curriculum provides a straightforward and effective recipe for enhancing general reasoning in LLMs. By strategically developing foundational reasoning skills in a data-rich and verifiable domain like math, and then systematically transferring and refining these skills across diverse tasks, LLMs can achieve broad and robust reasoning capabilities. For more in-depth information, you can read the full research paper here: Reasoning Curriculum: Bootstrapping Broad LLM Reasoning from Math.
