TLDR: The Encode–Think–Decode (ETD) method improves Large Language Model (LLM) reasoning by training models to recursively iterate over a small, reasoning-relevant subset of their layers during mid-training. This approach, which maintains the original model architecture and parameters, significantly boosts performance on complex tasks like mathematical reasoning (e.g., +28.4% on GSM8K, +36% on MATH for OLMo-2 1B). ETD also explores an adaptive depth strategy, allowing models to dynamically adjust computation per input token, leading to more efficient and targeted reasoning.
Large Language Models (LLMs) have shown incredible capabilities in tasks ranging from mathematics to logical puzzles. Traditionally, improving their reasoning power has involved either making models larger with more parameters and data, or using complex inference techniques like chain-of-thought prompting. However, a new research paper titled “Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughts” by Yeskendir Koishekenov, Aldo Lipani, and Nicola Cancedda introduces an innovative method called ETD that significantly boosts LLM reasoning without increasing model size or altering its core architecture.
The core idea behind Encode–Think–Decode (ETD) is to make LLMs “think” more deeply by recursively processing information within a specific set of layers. Interpretability studies have shown that the crucial computations for reasoning tasks are often concentrated in a limited range of layers within an LLM. ETD leverages this insight by training a base model to iterate over these reasoning-relevant layers during its mid-training phase. This approach amplifies the model’s latent reasoning abilities while keeping the original architecture, parameter count, hyperparameters, and training data exactly the same.
How ETD Works: Encode, Think, Decode
The ETD framework divides the LLM’s layers into three distinct groups:
- Latent Encoder (E): These initial layers are responsible for embedding the input data into a latent space and retrieving relevant information, setting the stage for deeper analysis.
- Recursive “Thinking” Block (T): This is the central unit where the iterative computation happens. The model cycles through these layers multiple times, generating and refining “latent thoughts” to perform multi-step reasoning.
- Latent Decoder (D): The final layers transform the refined latent representations back into the “language” space, ultimately producing the output tokens.
The researchers developed a practical method to identify which layers are most critical for reasoning. They used the Kneedle algorithm and analyzed the average angular distance between representations in consecutive layers. This helped them pinpoint the transition points where the rate of change in hidden representations slows down, indicating layers crucial for semantic integration and higher-level inference. For the OLMo-2 1B model, this analysis led to a configuration of 7 encoder layers, 4 recursive thinking layers, and 5 decoder layers (7-4*k-5, where ‘k’ is the number of iterations).
Impressive Performance Gains
The ETD method was integrated into the mid-training stage of the OLMo-2 1B Base model, a state-of-the-art open-source LLM. The model was evaluated across 17 reasoning benchmarks, categorized by increasing reasoning intensity, from factual knowledge to mathematical reasoning. The results were remarkable:
- On the GSM8K mathematical word problem benchmark, ETD achieved a substantial +28.4% relative accuracy improvement.
- For the challenging MATH dataset, it showed an even greater gain of +36% relative accuracy.
- Significant improvements were also observed in other categories like Reading Comprehension (+12.1%), Commonsense Reasoning (+4.8%), and Multi-Disciplinary Reasoning (+12.4%).
These gains were achieved by simply increasing the number of iterations (k) over the recursive “thinking” block, demonstrating that more latent reasoning directly leads to better performance on complex tasks. Interestingly, tasks relying primarily on memorization, like factual knowledge, showed only marginal benefits, further validating the targeted nature of ETD’s reasoning enhancement.
Adaptive Reasoning for Efficiency
Recognizing that not all problems require the same amount of reasoning, the paper also explores an adaptive depth strategy. Using an Adaptive Computation Time (ACT) mechanism, each token can dynamically determine how many times it needs to iterate through the recursive block. A lightweight “router” evaluates the hidden state after each iteration and decides whether more computation is necessary. This allows the model to allocate more processing steps to challenging tokens while bypassing further computation for simpler ones, leading to more efficient and adaptive use of resources. This adaptive approach even outperformed fixed-iteration ETD on some reading comprehension tasks while using fewer average iterations.
Also Read:
- MixReasoning: A Smart Approach to Efficient Language Model Thinking
- LightReasoner: How Smaller AI Models Can Enhance Larger Ones
A New Path for LLM Reasoning
ETD stands apart from previous recursive LLM approaches by focusing specifically on enhancing reasoning capabilities rather than just parameter efficiency. Its interpretability-guided selection of recursive layers, combined with its simplicity and validation on advanced open-source models, makes it a highly practical and effective method. By enabling LLMs to “think” recursively within their latent space, ETD offers a straightforward yet powerful way to strengthen their reasoning abilities without the need for larger models or external prompting techniques. For more details, you can read the full research paper here.
