TLDR: A new research paper investigates dual-architecture latent reasoning in LLMs, where a Base model communicates with a Coprocessor. The study tested two hypotheses to improve latent communication: increasing channel capacity and learning communication via joint finetuning. While both methods outperformed previous designs, a unified soft-embedding baseline achieved comparable results with fewer parameters, suggesting current dual designs primarily add compute rather than qualitatively improving reasoning. Latent space analysis revealed that latents tend to be redundant rather than specialized, indicating a need for new objectives and communication mechanisms to foster true System-2-like algorithmic planning.
Large Language Models (LLMs) have shown remarkable abilities in understanding and generating human-like text. However, their reasoning capabilities are often likened to “System 1” thinking – fast, intuitive, and heuristic-based, rather than the deliberate, analytical “System 2” reasoning that humans employ for complex problem-solving. This distinction, drawn from dual-process theories of cognition, highlights a key area for improvement in LLMs.
Traditionally, LLMs achieve multi-step reasoning through Chain-of-Thought (CoT) prompting, where intermediate steps are verbalized in natural language. While effective, CoT can be inefficient, consuming many tokens and limiting the abstraction of thought processes. This has led researchers to explore “latent reasoning,” an alternative where models perform internal, multi-step inference within their hidden states, only surfacing the final answer. This approach promises greater expressive bandwidth and a more compact representation of reasoning.
A recent study, “Exploring System 1 and 2 communication for latent reasoning in LLMs,” delves into dual-architecture latent reasoning. This design separates the fluent “Base” LLM (System 1) from a “Coprocessor” (System 2) that exchanges latent messages with the Base. The core question is whether LLM reasoning should reside in a separate module or within a single model’s forward pass.
Investigating Communication Channels
The researchers built upon previous work by Liu et al. (2024b) and proposed two main hypotheses to enhance latent communication between the Base and Coprocessor:
1. Hypothesis 1 (Frozen-Base KV augmentation): This approach aimed to increase the “channel capacity” by allowing the Coprocessor to write cache edits that reach all layers of the Base model, rather than just influencing the first hidden layer. The Base model’s parameters remained frozen during this process.
2. Hypothesis 2 (Co-finetuned dual-model): This hypothesis focused on learning communication through joint finetuning. Here, both the Base and Coprocessor models were trained together, enabling the Base to actively “listen” and adapt to the latent messages from the Coprocessor.
The study used smaller LLMs, GPT-2 (124M parameters) and Qwen-3 (0.6B parameters), to conduct experiments within a practical compute budget. They employed a strict three-pass training schedule to ensure a clear separation between next-token prediction and latent computation, preventing the Base model from shortcutting the intended latent reasoning.
Key Findings and Insights
The results showed that both Hypothesis 1 and Hypothesis 2 outperformed the original Liu et al. (2024b) dual-model design in terms of perplexity reduction and benchmark accuracy. Hypothesis 2, with its co-finetuned approach, consistently yielded the strongest gains, suggesting that allowing both modules to adapt to each other is crucial for effective cross-module communication.
However, a critical comparison was made against a “soft-embedding” baseline. This baseline involved a single LLM with the same forward pass and shared representations, but with an equivalent latent-token budget as the dual-model systems. Surprisingly, this unified soft-embedding model nearly matched Hypothesis 2 and surpassed Hypothesis 1. This finding suggests that the current dual-architecture designs might primarily be adding computational capacity rather than qualitatively improving reasoning through a distinct System-2 process.
Further evaluation on reasoning benchmarks like GSM8K, ProsQA, and a Countdown stress test revealed that simply scaling the latent-token budget beyond small values did not consistently improve robustness. In fact, for some tasks, increasing latent tokens even led to a dip in accuracy. This indicates that more latents don’t necessarily translate to better reasoning, but rather might just add more computational steps.
Latent Space Analysis
To understand why these dual architectures weren’t delivering the expected System-2-like reasoning, the researchers conducted an interpretability analysis of the latent spaces. They used diagnostics like cross-capture heatmaps and silhouette scores to assess whether different latents specialized in distinct intermediate computations or if they were redundant.
The analysis revealed that latents tended towards redundant computations. Their representational subspaces were highly overlapping, meaning extra latents mostly amplified confidence rather than adding new algorithmic structure or specialized reasoning steps. While fine-tuning helped reduce some directional redundancy, it didn’t lead to clear cluster separation or specialization among latents.
Also Read:
- The Dual Strengths of AI Reasoning: Chain-of-Thought for Approximation, Latent Thought for Parallelism
- Unpacking AI’s Inner Workings: How Training Shapes Reasoning Circuits
Future Directions for Latent Reasoning
The study concludes that while dual-model latent reasoning remains promising in principle, current instantiations do not yet create the conditions for genuine System-2-like computation to emerge. The benefits observed largely reflect added capacity rather than structured reasoning.
The paper suggests that future work should focus on designing objectives and communication mechanisms that explicitly shape latent spaces for algorithmic planning. This could involve rewarding diversity or orthogonality in latent representations to encourage broader search, or developing training schedules that preserve language competence while gradually fostering multi-step reasoning. For more details, you can read the full research paper here.
