TLDR: Klear-Reasoner is a new AI model that excels in math and programming reasoning. It achieves high performance by using a unique training approach that combines high-quality data for initial fine-tuning and a novel reinforcement learning technique called GPPO. GPPO improves exploration and learning from mistakes by carefully handling gradient signals, leading to more stable and effective training. The model demonstrates state-of-the-art performance on challenging benchmarks.
A new technical report introduces Klear-Reasoner, an advanced AI model designed to significantly improve long-form reasoning capabilities in complex tasks like mathematics and programming. This model demonstrates a careful and deliberate approach to problem-solving, achieving impressive results across various benchmarks.
The development of high-performance reasoning models often faces challenges due to the incomplete disclosure of training details from previous works. Klear-Reasoner aims to address this by providing an in-depth analysis of its entire post-training workflow, from data preparation and supervised fine-tuning to reinforcement learning, along with detailed studies of each experimental component.
Supervised Fine-Tuning: Quality Over Quantity
For the initial supervised fine-tuning (SFT) phase, the Klear-Reasoner team adopted a quality-centric data construction strategy. Their experiments revealed that a smaller number of high-quality data sources are more effective than a large, diverse collection. This is because high-quality examples ensure consistent learning of accurate reasoning patterns. Interestingly, the research also found that including difficult samples, even if not perfectly accurate, can lead to better results by promoting the model’s exploration capacity.
The choice of ‘teacher’ model for generating responses during SFT also proved crucial. Using a stronger teacher model, like DeepSeek-R1-0528, resulted in higher student model performance, underscoring the importance of teacher quality in knowledge distillation.
Gradient-Preserving Clipping Policy Optimization (GPPO)
A core innovation in Klear-Reasoner’s reinforcement learning (RL) phase is the introduction of Gradient-Preserving clipping Policy Optimization (GPPO). Traditional clipping mechanisms in RL, which control the magnitude of policy updates, often suppress critical exploration signals and ignore valuable information from suboptimal attempts. This can lead to premature termination of exploration and slower learning from mistakes.
GPPO tackles these issues by gently backpropagating gradients from ‘clipped’ tokens – those that would normally be discarded. This means that even for actions that fall outside the typical clipping range, their gradient information is still used in the learning process. This approach not only enhances the model’s ability to explore new solutions but also significantly improves its efficiency in learning from negative examples, allowing for faster trial-and-error adjustments.
The report highlights that GPPO maintains training stability while providing these benefits, striking a balance between aggressive exploration and controlled updates. This is a key differentiator from other methods that might discard or overly restrict gradient signals.
Refined RL Training and Reward Design
Klear-Reasoner’s RL training also incorporates several other refined techniques. It uses a joint training approach that includes a language model (LM) loss for positive examples, which helps improve the utilization efficiency of good samples and acts as a regularizer to maintain a reasonable output distribution.
For reward design, the model uses a binary system for mathematical tasks (correct or incorrect) but introduces a ‘soft reward’ mechanism for coding tasks. Instead of just a pass/fail, code solutions receive a proportional reward based on the percentage of test cases they pass. This granular feedback helps mitigate sparse rewards, preserves learning signals from partially correct solutions, and encourages incremental improvements.
Furthermore, the team implemented data filtering strategies for code problems to remove examples with flawed test cases, ensuring a more reliable reinforcement learning signal. They also found that filtering out ‘zero-advantage’ groups (where all responses are either perfectly correct or incorrect) during training helps maintain clearer optimization signals and improves generalization.
Also Read:
- Enhancing LLM Code Generation with Klear-CodeTest’s Advanced Test Case Synthesis
- THINK TUNING: A New Approach to Teaching LLMs How to Self-Reflect
Outstanding Performance
The integration of quality-centric SFT and GPPO RL has enabled Klear-Reasoner-8B to achieve exceptional reasoning abilities. The model scored 90.5% on AIME 2024, 83.2% on AIME 2025, 66.0% on LiveCodeBench V5, and 58.1% on LiveCodeBench V6. These results demonstrate that Klear-Reasoner consistently matches or surpasses state-of-the-art models of comparable scale, even outperforming models trained with significantly more data through the power of its RL fine-tuning.
The work underscores that principled data curation, targeted supervised fine-tuning, and carefully designed reinforcement learning optimization can collectively lead to substantial advancements in long-form reasoning performance. For more technical details, you can refer to the full research paper here.
