TLDR: The Adaptive Dual Reasoner (ADR) is a new framework for Large Reasoning Models (LRMs) that addresses the problem of “overthinking” by dynamically switching between fast and slow reasoning modes based on task complexity. Trained through supervised fine-tuning and reinforcement learning with Entropy-guided Hybrid Policy Optimization (EHPO), ADR significantly improves reasoning performance while drastically reducing output length on mathematical benchmarks, achieving a better balance between accuracy and efficiency.
Large Reasoning Models (LRMs) have shown impressive capabilities in solving complex problems, from intricate mathematical equations to logical puzzles. However, their power often comes at a cost: they tend to “overthink,” generating lengthy and sometimes redundant reasoning steps. This overthinking leads to increased computational expenses and slower response times, limiting their practical application.
To address this challenge, researchers have introduced a novel approach called the Adaptive Dual Reasoner (ADR). This innovative framework equips LRMs with two distinct reasoning modes: a “fast thinking” mode for straightforward tasks and a “slow thinking” mode for more complex, demanding problems. The brilliance of ADR lies in its ability to dynamically switch between these modes, adapting its reasoning effort based on the complexity of the task at hand.
The development of ADR involves a two-stage training process. The first stage, known as the cold-start stage, uses supervised fine-tuning (SFT). During this phase, the model learns to integrate both fast and slow reasoning modes. This is achieved by constructing a specialized hybrid reasoning dataset, which provides extensive examples of how to apply both types of thinking. This initial training gives the model the foundational ability to recognize and utilize different reasoning styles.
The second stage focuses on optimizing the reasoning effort through reinforcement learning. Here, a framework called Entropy-guided Hybrid Policy Optimization (EHPO) is employed. EHPO is designed to refine how ADR allocates its cognitive resources. It uses an entropy-guided dynamic rollout strategy, which allows the model to explore multiple reasoning paths when faced with high-uncertainty (high-entropy) situations, typically indicating a transition from an easy to a hard problem. Additionally, a difficulty-aware penalty helps balance the use of fast and slow reasoning, ensuring efficiency without sacrificing accuracy.
EHPO’s reward system is carefully designed with four signals: format compliance, accuracy, unit semantic correctness, and mode control. The unit semantic reward encourages the model to correctly classify reasoning steps as either easy (without reflection keywords) or hard (with reflection keywords like “Wait” or “However”). The mode control reward incentivizes the model to use the easy mode for simpler tasks and the hard mode for more challenging ones, optimizing resource allocation.
Experiments on challenging mathematical reasoning benchmarks, such as AIME2025, AIME2024, and MATH500, have demonstrated ADR’s effectiveness. It achieves a remarkable balance between reasoning performance and efficiency. For instance, ADR showed a performance gain of up to 6.1% on AIME2024, while simultaneously reducing the reasoning output length by a substantial 49.5% to 59.3% across various tasks. This means the model solves problems more accurately and with significantly fewer unnecessary steps.
A key component, the Entropy-guided Dynamic Rollout (EDR) strategy, proved crucial. Without EDR, the benefits of the reinforcement learning stage were limited. With EDR, the model’s accuracy and efficiency significantly improved, confirming that this strategy enables more effective trade-offs between accuracy and efficiency by expanding the exploration space when needed.
Also Read:
- Enhancing LLM Reasoning with Attribution-Based Credit Assignment and Dynamic Exploration
- Multidimensional Feedback for Smarter Language Models
In essence, ADR represents a significant step forward in making large reasoning models more efficient and practical. By allowing models to adaptively switch between different thinking speeds, it ensures that computational resources are allocated strategically, leading to faster, more accurate, and less verbose reasoning. You can read the full research paper here: Adaptive Dual Reasoner: Large Reasoning Models Can Think Efficiently by Hybrid Reasoning.
