TLDR: A new research paper introduces ‘step entropy,’ a novel metric to identify and remove redundant steps in Large Language Models’ (LLMs) Chain-of-Thought (CoT) reasoning processes. By pruning up to 80% of low-entropy steps, the method achieves significant token reductions (16-57%) and improves LLM inference efficiency while maintaining or enhancing accuracy across various models and mathematical benchmarks. A two-stage training strategy enables LLMs to autonomously generate these compressed thought chains, making them more practical for real-world applications.
Large Language Models (LLMs) have shown incredible abilities in solving complex problems, especially when they use a technique called Chain-of-Thought (CoT). This method involves the LLM breaking down a problem into a series of explicit, intermediate reasoning steps, much like how a human might think through a problem slowly. This ‘slow thinking’ helps models like DeepSeek-R1 and Qwen3 achieve better performance on multi-step tasks in areas like mathematics, coding, and logic.
However, this detailed thinking comes with a significant drawback: verbosity. The generated thought processes often contain a lot of redundant information, leading to higher computing costs, increased time for the model to respond, and overall inefficiency. As LLMs become larger and are used more widely, these inefficiencies become a major bottleneck for practical applications.
Previous research has tried to tackle this by making the CoT process implicit or by compressing the reasoning chain at different levels, from individual words to larger chunks. While these methods improve efficiency, they haven’t offered a clear, principled way to identify and remove entire reasoning steps that are truly unnecessary.
This is where a new research paper, Compressing Chain-of-Thought in LLMs via Step Entropy, introduces a novel solution. The authors, Zeju Li, Jianyuan Zhong, Ziyang Zheng, Xiangyu Wen, Zhijian Xu, Yingying Cheng, Fan Zhang, and Qiang Xu, propose a new metric called ‘step entropy’. This metric quantifies how much useful information each individual reasoning step contributes to the final answer. Their core idea is that steps with low entropy are more predictable and less informative, meaning they are likely redundant and can be safely removed without harming the accuracy of the final answer.
The method works in a straightforward way. First, the LLM generates a complete Chain-of-Thought. Then, for each step in this thought process, its ‘step entropy’ is calculated. Steps are then ranked by their entropy scores, and those with the lowest entropy are identified for pruning. The paper demonstrates that an astonishing 80% of these low-entropy intermediate steps can be removed with only minor impact on the final answer’s accuracy. This is a stark contrast to removing steps randomly or those with high entropy, which severely damages the model’s reasoning performance.
The empirical validation of this approach is compelling. Experiments on mathematical reasoning benchmarks using models like DeepSeek-R1 (7B and 14B) and Qwen3-8B showed significant reductions in the number of ‘thinking tokens’ – ranging from 16% to 57% – while maintaining or even improving accuracy. This cross-model consistency highlights that step entropy is a robust and widely applicable principle for identifying redundancy, regardless of the specific LLM architecture.
Furthermore, the researchers found that pruning entire reasoning steps is far more effective than pruning individual tokens. Removing single low-entropy tokens can break the meaning and structure of a critical step, making it incomprehensible. In contrast, removing an entire low-entropy step preserves the integrity of the remaining, more important steps, leading to a much more reliable compression strategy.
To make LLMs capable of generating compressed CoTs on their own, the paper proposes a two-stage training strategy. The first stage, Supervised Fine-Tuning (SFT), teaches the model to predict when to use a special ‘[SKIP]’ token based on pre-compressed training data. The second stage uses Group Relative Policy Optimization (GRPO), a reinforcement learning technique. Here, the model learns to optimize a reward function that balances accuracy with compression efficiency, encouraging it to strategically decide when to skip redundant reasoning steps during inference.
Also Read:
- Fine-Grained Reward Signals for Large Language Models
- ThinkingF: A New Approach to Automate Mathematical Formalization with AI
This innovative framework has profound implications for the practical deployment of LLMs. By significantly enhancing inference efficiency while rigorously preserving accuracy, it makes powerful reasoning models more sustainable and accessible. It also offers new insights into the underlying structure of reasoning processes within LLMs, suggesting that current models often generate a substantial amount of unnecessary detail in their thought chains. This understanding can inform the design of future LLMs, potentially leading to even more inherently efficient reasoning architectures.
